Computing interface system

ABSTRACT

Computing interface systems and methods are disclosed. Some implementations include a first accelerometer attached to a first fastening article that is capable of holding the first accelerometer in place on a portion of a thumb of a user. Some implementations may also include a second accelerometer attached to a second fastening article that is capable of holding the second accelerometer in place on a portion of a wrist of a user. Some implementations may additionally or alternatively include magnetometers and/or gyroscopes attached to the first and second fastening articles. Some implementations may also include a processing device configured to receive measurements from the accelerometers, magnetometers, and/or gyroscopes and identify, based on the measurements, symbols associated with motions of a user&#39;s hand and/or the orientation of the hand. Some implementations may allow a user to control a cursor in a three dimensional virtual space and interact with objects in that space.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. patent application Ser. No.13/804,871 filed on Mar. 14, 2013 and U.S. Provisional PatentApplication No. 61/729,573, which was filed on Nov. 24, 2012. The entirecontents of these applications are hereby incorporated by reference inthe specification of this application.

TECHNICAL FIELD

This disclosure relates to systems and methods for human-computerinteraction through hand motions.

BACKGROUND

Computing devices are often equipped with one or more interfaces forinterpreting hand motions of a user. For decades the Qwerty keyboard andmouse were the dominant interface devices in the computing field. Morerecently, the touch screen interface has gained popularity. Anotherrelatively recent development has been the use of hand-held gamingcontrollers that use accelerometers to detect hand motions of a user.

SUMMARY

In a first aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a firstaccelerometer and a first fastening article attached to the firstaccelerometer, the first fastening article capable of holding the firstaccelerometer in place on a portion of a thumb of a user. The systemsmay include a second accelerometer and a second fastening articleattached to the second accelerometer, the second fastening articlecapable of holding the second accelerometer in place on a portion of awrist of a user.

This and other aspects can each optionally include one or more of thefollowing features. The first fastening article may be a ring band. Thesecond fastening article may be a wrist band. A system may include firstmagnetometer attached to the first fastening article and a secondmagnetometer attached to the second fastening article. A system mayinclude a processing device configured to receive accelerationmeasurements from the first accelerometer and determine, based at leastin part on the received acceleration measurements, when a thumb of auser wearing the first fastening article and the second fasteningarticle has been tapped against a surface. The processing device may befurther configured to receive acceleration measurements from the secondaccelerometer, receive magnetic flux measurements from the firstmagnetometer and from the second magnetometer, and when a tap of thethumb is detected, determine, based at least in part on accelerationmeasurements from the first accelerometer and the second accelerometerand on magnetic flux measurements from the first magnetometer and thesecond magnetometer, which tap target from among a set of tap targetslocated on fingers of a hand of the user have been tapped by the thumb.The processing device may be further configured to map the tap of thetap target to a symbol associated with the tap target. A system mayinclude a first gyroscope attached to the first fastening article and asecond gyroscope attached to the second fastening article. A system mayinclude a processing device configured to receive accelerationmeasurements from the first accelerometer and determine, based at leastin part on the received acceleration measurements, when a thumb of auser wearing the first fastening article and the second fasteningarticle has been tapped against a surface. The processing device may befurther configured to receive acceleration measurements from the secondaccelerometer, receive angular rate measurements from the firstgyroscope and from the second gyroscope, and when a tap of the thumb isdetected, determine, based at least in part on acceleration measurementsfrom the first accelerometer and the second accelerometer and on angularrate measurements from the first gyroscope and the second gyroscope,which tap target from among a set of tap targets located on fingers of ahand of the user have been tapped by the thumb. The processing devicemay be further configured to map the tap of the tap target to a symbolassociated with the tap target. A system may include two or more wiresthat connect the first fastening device to the second fastening device.A system may include a wireless transmitter attached to the secondfastening article, and the wireless transmitter may be configured totransmit data based on acceleration measurements from the secondaccelerometer to a computing device. A system may include a wirelesstransmitter attached to the second fastening article, and the wirelesstransmitter may be configured to transmit data based on accelerationmeasurements from the first accelerometer to a computing device. Asystem may include a processing device configured to receiveacceleration measurements from the first accelerometer and determine,based at least in part on the received acceleration measurements, when athumb of a user wearing the first fastening article and the secondfastening article has been tapped against a surface. The processingdevice may be further configured to receive acceleration measurementsfrom the second accelerometer and when a tap of the thumb is detected,determine, based at least in part on acceleration measurements from thefirst accelerometer and the second accelerometer, which tap target fromamong a set of tap targets located on fingers of a hand of the user havebeen tapped by the thumb. The processing device may be furtherconfigured to map the tap of the tap target to a symbol associated withthe tap target. A system may include a display device configured topresent images produced by the processing device and the images mayinclude graphic representations of symbols selected based the data. Thedisplay device may be a liquid crystal display. The display device maybe a projector. The processing device may be attached to the secondfastening article. The processing device may be part of a remotecomputing device.

In a second aspect, in general, the subject matter described in thisspecification can be embodied in methods that include receiving a firstset of acceleration measurements from a first accelerometer that isattached to a thumb of a user. The methods may include receiving asecond set of acceleration measurements from a second accelerometer thatis attached to a wrist of the user. The methods may include detecting,based at least in part on the first set of acceleration measurements,when the thumb is tapped on a surface. The methods may include when atap of the thumb is detected, determining, based at least in part on thefirst set of acceleration measurements and the second set ofacceleration measurements, an orientation of the first accelerometerrelative to the second accelerometer. The methods may includeidentifying, based on the orientation, a tap target, from among aplurality of tap targets located on fingers of the user, that wastouched during the tap. The methods may include assigning a symbolassociated with the tap target to the tap. The methods may includetransmitting, storing, or displaying the symbol.

This and other aspects can each optionally include one or more of thefollowing features. A first set of magnetic flux measurements may bereceived from a first magnetometer that is attached to the thumb of theuser. A second set of magnetic flux measurements may be received from asecond magnetometer that is attached to the wrist of the user. Theorientation may be determined based at least in part on the first set ofmagnetic flux measurements and the second set of magnetic fluxmeasurements. Determining the orientation may include filtering thefirst set acceleration measurements to determine a first estimate ofacceleration experienced by the first accelerometer while the thumb wasat rest with respect to a hand of the user on the tap target; filteringthe second set of acceleration measurements to determine a secondestimate of acceleration experienced by the second accelerometer whilethe thumb was at rest with respect to the hand of the user on the taptarget; filtering the first set magnetic flux measurements to determinea first estimate of magnetic flux experienced by the first magnetometerwhile the thumb was at rest with respect to the hand of the user on thetap target; filtering the second set of magnetic flux measurements todetermine a second estimate of magnetic flux experienced by the secondaccelerometer while the thumb was at rest with respect to the hand ofthe user on the tap target; and determining a rotation that willapproximately align the first estimate of acceleration with the secondestimate of acceleration and that also will approximately align thefirst estimate of magnetic flux with the second estimate of magneticflux. A first set of angular rate measurements may be received from afirst gyroscope that is attached to the thumb of the user. A second setof angular rate measurements may be received from a second gyroscopethat is attached to the wrist of the user. The orientation may bedetermined based at least in part on first set of angular ratemeasurements and the second set of angular rate measurements.Determining the orientation may include integrating the first set ofacceleration measurements and the first set of angular rate measurementsto determine a first estimate of an orientation, with respect to areference position, of the first accelerometer at a time during the tap;integrating the second set of acceleration measurements and the secondset of angular rate measurements to determine a second estimate of anorientation, with respect to a reference position, of the secondaccelerometer at the time during the tap; and comparing the firstestimate of the orientation of the first accelerometer to the secondestimate of the orientation of the second accelerometer. An estimate maybe determined, based on the second set of acceleration measurements, ofan orientation of an acceleration experienced by the secondaccelerometer with respect to the axes of the second accelerometerduring the tap. Assigning the symbol to the tap may include selectingthe symbol from among a plurality of symbols associated with the taptarget based on the orientation of the acceleration experienced by thesecond accelerometer. The acceleration experienced by the secondaccelerometer may be dominated by acceleration caused by thegravitational force of the Earth and the selection may be made based atleast in part on an estimate of an angle between the gravitationalacceleration and an axis running approximately parallel to a forearm ofthe user. The user may be enabled to indicate a choice from among theplurality of the symbols associated with the tap target, by adjustingthe angle of wrist or the angle of a forearm of the user with respect tothe gravitational force of the Earth during the tap. The user may beenabled to turn a palm side of the wrist down to face towards the Earthto enter a cursor manipulation mode, in which acceleration measurementsfrom the second accelerometer are used to move a cursor in a virtualspace. Determining the orientation may include filtering the first setof acceleration measurements to determine an estimate of a decelerationvector caused by the impact of the thumb with the tap target.

In a third aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a data processingapparatus and a data storage device storing instructions executable bythe data processing apparatus that upon execution by the data processingapparatus cause the data processing apparatus to perform operationsincluding receiving a first set of acceleration measurements from afirst accelerometer that is attached to a thumb of a user. Theoperations may include receiving a second set of accelerationmeasurements from a second accelerometer that is attached to a wrist ofthe user. The operations may include detecting, based at least in parton the first set of acceleration measurements, when the thumb is tappedon a surface. The operations may include when a tap of the thumb isdetected, determining, based at least in part on the first set ofacceleration measurements and the second set of accelerationmeasurements, an orientation of the first accelerometer relative to thesecond accelerometer. The operations may include identifying, based onthe orientation, a tap target, from among a plurality of tap targetslocated on fingers of the user, that was touched during the tap. Theoperations may include assigning a symbol associated with the tap targetto the tap. The operations may include transmitting, storing, ordisplaying the symbol.

This and other aspects can each optionally include one or more of thefollowing features. A first set of magnetic flux measurements may bereceived from a first magnetometer that is attached to the thumb of theuser. A second set of magnetic flux measurements may be received from asecond magnetometer that is attached to the wrist of the user. Theorientation may be determined based at least in part on the first set ofmagnetic flux measurements and the second set of magnetic fluxmeasurements. Determining the orientation may include filtering thefirst set acceleration measurements to determine a first estimate ofacceleration experienced by the first accelerometer while the thumb wasat rest with respect to a hand of the user on the tap target; filteringthe second set of acceleration measurements to determine a secondestimate of acceleration experienced by the second accelerometer whilethe thumb was at rest with respect to the hand of the user on the taptarget; filtering the first set magnetic flux measurements to determinea first estimate of magnetic flux experienced by the first magnetometerwhile the thumb was at rest with respect to the hand of the user on thetap target; filtering the second set of magnetic flux measurements todetermine a second estimate of magnetic flux experienced by the secondaccelerometer while the thumb was at rest with respect to the hand ofthe user on the tap target; and determining a rotation that willapproximately align the first estimate of acceleration with the secondestimate of acceleration and that also will approximately align thefirst estimate of magnetic flux with the second estimate of magneticflux. A first set of angular rate measurements may be received from afirst gyroscope that is attached to the thumb of the user. A second setof angular rate measurements may be received from a second gyroscopethat is attached to the wrist of the user. The orientation may bedetermined based at least in part on first set of angular ratemeasurements and the second set of angular rate measurements.Determining the orientation may include integrating the first set ofacceleration measurements and the first set of angular rate measurementsto determine a first estimate of an orientation, with respect to areference position, of the first accelerometer at a time during the tap;integrating the second set of acceleration measurements and the secondset of angular rate measurements to determine a second estimate of anorientation, with respect to a reference position, of the secondaccelerometer at the time during the tap; and comparing the firstestimate of the orientation of the first accelerometer to the secondestimate of the orientation of the second accelerometer. An estimate maybe determined, based on the second set of acceleration measurements, ofan orientation of an acceleration experienced by the secondaccelerometer with respect to the axes of the second accelerometerduring the tap. Assigning the symbol to the tap may include selectingthe symbol from among a plurality of symbols associated with the taptarget based on the orientation of the acceleration experienced by thesecond accelerometer. The acceleration experienced by the secondaccelerometer may be dominated by acceleration caused by thegravitational force of the Earth and the selection may be made based atleast in part on an estimate of an angle between the gravitationalacceleration and an axis running approximately parallel to a forearm ofthe user. The user may be enabled to indicate a choice from among theplurality of the symbols associated with the tap target, by adjustingthe angle of wrist or the angle of a forearm of the user with respect tothe gravitational force of the Earth during the tap. The user may beenabled to turn a palm side of the wrist down to face towards the Earthto enter a cursor manipulation mode, in which acceleration measurementsfrom the second accelerometer are used to move a cursor in a virtualspace. Determining the orientation may include filtering the first setof acceleration measurements to determine an estimate of a decelerationvector caused by the impact of the thumb with the tap target.

In a fourth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a wireless networkinterface configured to receive a first set of acceleration measurementsfrom a first accelerometer that is attached to a thumb of a user. Thesystems may include a wireless network interface configured to receive asecond set of acceleration measurements from a second accelerometer thatis attached to a wrist of the user. The systems may include a moduleconfigured to detect, based at least in part on the first set ofacceleration measurements, when the thumb is tapped on a surface. Thesystems may include a module configured to, when a tap of the thumb isdetected, determine, based at least in part on the first set ofacceleration measurements and the second set of accelerationmeasurements, an orientation of the first accelerometer relative to thesecond accelerometer. The systems may include a module configured toidentify, based on the orientation, a tap target, from among a pluralityof tap targets located on fingers of the user, that was touched duringthe tap. The systems may include a module configured to assign a symbolassociated with the tap target to the tap. The systems may include amodule configured to transmit, store, or display the symbol.

This and other aspects can each optionally include one or more of thefollowing features. A first set of magnetic flux measurements may bereceived from a first magnetometer that is attached to the thumb of theuser. A second set of magnetic flux measurements may be received from asecond magnetometer that is attached to the wrist of the user. Theorientation may be determined based at least in part on the first set ofmagnetic flux measurements and the second set of magnetic fluxmeasurements. Determining the orientation may include filtering thefirst set acceleration measurements to determine a first estimate ofacceleration experienced by the first accelerometer while the thumb wasat rest with respect to a hand of the user on the tap target; filteringthe second set of acceleration measurements to determine a secondestimate of acceleration experienced by the second accelerometer whilethe thumb was at rest with respect to the hand of the user on the taptarget; filtering the first set magnetic flux measurements to determinea first estimate of magnetic flux experienced by the first magnetometerwhile the thumb was at rest with respect to the hand of the user on thetap target; filtering the second set of magnetic flux measurements todetermine a second estimate of magnetic flux experienced by the secondaccelerometer while the thumb was at rest with respect to the hand ofthe user on the tap target; and determining a rotation that willapproximately align the first estimate of acceleration with the secondestimate of acceleration and that also will approximately align thefirst estimate of magnetic flux with the second estimate of magneticflux. A first set of angular rate measurements may be received from afirst gyroscope that is attached to the thumb of the user. A second setof angular rate measurements may be received from a second gyroscopethat is attached to the wrist of the user. The orientation may bedetermined based at least in part on first set of angular ratemeasurements and the second set of angular rate measurements.Determining the orientation may include integrating the first set ofacceleration measurements and the first set of angular rate measurementsto determine a first estimate of an orientation, with respect to areference position, of the first accelerometer at a time during the tap;integrating the second set of acceleration measurements and the secondset of angular rate measurements to determine a second estimate of anorientation, with respect to a reference position, of the secondaccelerometer at the time during the tap; and comparing the firstestimate of the orientation of the first accelerometer to the secondestimate of the orientation of the second accelerometer.

An estimate may be determined, based on the second set of accelerationmeasurements, of an orientation of an acceleration experienced by thesecond accelerometer with respect to the axes of the secondaccelerometer during the tap. Assigning the symbol to the tap mayinclude selecting the symbol from among a plurality of symbolsassociated with the tap target based on the orientation of theacceleration experienced by the second accelerometer. The accelerationexperienced by the second accelerometer may be dominated by accelerationcaused by the gravitational force of the Earth and the selection may bemade based at least in part on an estimate of an angle between thegravitational acceleration and an axis running approximately parallel toa forearm of the user. The user may be enabled to indicate a choice fromamong the plurality of the symbols associated with the tap target, byadjusting the angle of wrist or the angle of a forearm of the user withrespect to the gravitational force of the Earth during the tap. The usermay be enabled to turn a palm side of the wrist down to face towards theEarth to enter a cursor manipulation mode, in which accelerationmeasurements from the second accelerometer are used to move a cursor ina virtual space. Determining the orientation may include filtering thefirst set of acceleration measurements to determine an estimate of adeceleration vector caused by the impact of the thumb with the taptarget.

In a fifth aspect, in general, the subject matter described in thisspecification can be embodied in one or more computer-readable storagemedia encoded with a computer program. The computer program may includeinstructions that when executed by data a processing apparatus cause thedata processing apparatus to perform operations including receiving afirst set of acceleration measurements from a first accelerometer thatis attached to a thumb of a user. The operations may include receiving asecond set of acceleration measurements from a second accelerometer thatis attached to a wrist of the user. The operations may includedetecting, based at least in part on the first set of accelerationmeasurements, when the thumb is tapped on a surface. The operations mayinclude when a tap of the thumb is detected, determining, based at leastin part on the first set of acceleration measurements and the second setof acceleration measurements, an orientation of the first accelerometerrelative to the second accelerometer. The operations may includeidentifying, based on the orientation, a tap target, from among aplurality of tap targets located on fingers of the user, that wastouched during the tap. The operations may include assigning a symbolassociated with the tap target to the tap. The operations may includetransmitting, storing, or displaying the symbol.

This and other aspects can each optionally include one or more of thefollowing features. A first set of magnetic flux measurements may bereceived from a first magnetometer that is attached to the thumb of theuser. A second set of magnetic flux measurements may be received from asecond magnetometer that is attached to the wrist of the user. Theorientation may be determined based at least in part on the first set ofmagnetic flux measurements and the second set of magnetic fluxmeasurements. Determining the orientation may include filtering thefirst set acceleration measurements to determine a first estimate ofacceleration experienced by the first accelerometer while the thumb wasat rest with respect to a hand of the user on the tap target; filteringthe second set of acceleration measurements to determine a secondestimate of acceleration experienced by the second accelerometer whilethe thumb was at rest with respect to the hand of the user on the taptarget; filtering the first set magnetic flux measurements to determinea first estimate of magnetic flux experienced by the first magnetometerwhile the thumb was at rest with respect to the hand of the user on thetap target; filtering the second set of magnetic flux measurements todetermine a second estimate of magnetic flux experienced by the secondaccelerometer while the thumb was at rest with respect to the hand ofthe user on the tap target; and determining a rotation that willapproximately align the first estimate of acceleration with the secondestimate of acceleration and that also will approximately align thefirst estimate of magnetic flux with the second estimate of magneticflux. A first set of angular rate measurements may be received from afirst gyroscope that is attached to the thumb of the user. A second setof angular rate measurements may be received from a second gyroscopethat is attached to the wrist of the user. The orientation may bedetermined based at least in part on first set of angular ratemeasurements and the second set of angular rate measurements.Determining the orientation may include integrating the first set ofacceleration measurements and the first set of angular rate measurementsto determine a first estimate of an orientation, with respect to areference position, of the first accelerometer at a time during the tap;integrating the second set of acceleration measurements and the secondset of angular rate measurements to determine a second estimate of anorientation, with respect to a reference position, of the secondaccelerometer at the time during the tap; and comparing the firstestimate of the orientation of the first accelerometer to the secondestimate of the orientation of the second accelerometer. An estimate maybe determined, based on the second set of acceleration measurements, ofan orientation of an acceleration experienced by the secondaccelerometer with respect to the axes of the second accelerometerduring the tap. Assigning the symbol to the tap may include selectingthe symbol from among a plurality of symbols associated with the taptarget based on the orientation of the acceleration experienced by thesecond accelerometer. The acceleration experienced by the secondaccelerometer may be dominated by acceleration caused by thegravitational force of the Earth and the selection may be made based atleast in part on an estimate of an angle between the gravitationalacceleration and an axis running approximately parallel to a forearm ofthe user. The user may be enabled to indicate a choice from among theplurality of the symbols associated with the tap target, by adjustingthe angle of wrist or the angle of a forearm of the user with respect tothe gravitational force of the Earth during the tap. The user may beenabled to turn a palm side of the wrist down to face towards the Earthto enter a cursor manipulation mode, in which acceleration measurementsfrom the second accelerometer are used to move a cursor in a virtualspace. Determining the orientation may include filtering the first setof acceleration measurements to determine an estimate of a decelerationvector caused by the impact of the thumb with the tap target.

In a sixth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a ring band, afirst accelerometer attached to the ring band, a first magnetometerattached to the ring band, a wrist band, a second accelerometer attachedto the wrist band, and a second magnetometer attached to the wrist band.

This and other aspects can each optionally include one or more of thefollowing features. The systems may include a gyroscope attached to thewrist band. The systems may include a gyroscope attached to the ringband. The systems may include a wireless transmitter attached to thewrist band. The systems may include a wireless transmitter attached tothe ring band. The systems may include two or more wire connecting thering band to the wrist band. The systems may include a processing deviceattached to the wrist band. The processing device may be amicro-controller. The processing device may be a microprocessor. Thesystems may include a processing device attached to the ring band.

In a seventh aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a ring band, afirst accelerometer attached to the ring band, a first magnetometerattached to the ring band, a wrist band, a second accelerometer attachedto the wrist band, a second magnetometer attached to the wrist band, anda processor configured to receive sensor readings from the firstaccelerometer, the first magnetometer, the second accelerometer, and thesecond magnetometer and to detect hand gestures based on these receivedsensor readings.

This and other aspects can each optionally include one or more of thefollowing features. Detecting hand gestures may include calculating theorientation of the first accelerometer relative to the secondaccelerometer. Detecting hand gestures may include calculating theorientation of the first accelerometer relative to the secondaccelerometer. Detecting hand gestures may include calculating theorientation of the first magnetometer relative to the secondmagnetometer.

In an eighth aspect, in general, the subject matter described in thisspecification can be embodied in methods that include attaching a firstsensor component to a first body part so that the sensor component isheld in a fixed orientation with respect to the first body part andattaching a second sensor component to a second body part so that thesensor component is held in a fixed orientation with respect to thesecond body part. The methods may include receiving signal from thefirst sensor component reflecting readings from a sensor in the firstsensor component and receiving signal from the second sensor componentreflecting readings from a sensor in the second sensor component. Themethods may include estimating the relative orientation of the firstsensor component and the second sensor component based in part on thereadings from the sensors in the first and second sensor components.

This and other aspects can each optionally include one or more of thefollowing features. The first sensor component may include anaccelerometer and a magnetometer, the second sensor component mayinclude an accelerometer and a magnetometer, and the sensor readings mayinclude acceleration measurements and magnetic flux measurements fromthe first sensor component and the second sensor component. The firstsensor component may include an accelerometer and a gyroscope, thesecond sensor component may include an accelerometer and a gyroscope,and the sensor readings may include acceleration measurements andangular rate measurements from the first sensor component and the secondsensor component.

In a ninth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a fasteningarticle for attaching a first sensor component to a first body part sothat the sensor component is held in a fixed orientation with respect tothe first body part and a fastening article for attaching a secondsensor component to a second body part so that the sensor component isheld in a fixed orientation with respect to the second body part. Thesystems may include a wireless networking interface for receiving signalfrom the first sensor component reflecting readings from a sensor in thefirst sensor component and receiving signal from the second sensorcomponent reflecting readings from a sensor in the second sensorcomponent. The systems may include a module configured to estimate therelative orientation of the first sensor component and the second sensorcomponent based in part on the readings from the sensors in the firstand second sensor components.

This and other aspects can each optionally include one or more of thefollowing features. The first sensor component may include anaccelerometer and a magnetometer, the second sensor component mayinclude an accelerometer and a magnetometer, and the sensor readings mayinclude acceleration measurements and magnetic flux measurements fromthe first sensor component and the second sensor component. The firstsensor component may include an accelerometer and a gyroscope, thesecond sensor component may include an accelerometer and a gyroscope,and the sensor readings may include acceleration measurements andangular rate measurements from the first sensor component and the secondsensor component.

In a tenth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a ring band, afirst accelerometer attached to the ring band, a fastening articleconfigured to hold an object in a fixed orientation with respect to thehand or forearm of a user, and a second accelerometer attached to afastening article.

This and other aspects can each optionally include one or more of thefollowing features. The systems may include a gyroscope attached to thefirst fastening article. The systems may include a gyroscope attached tothe second fastening article. The systems may include a wirelesstransmitter attached to the first fastening article. The systems mayinclude a wireless transmitter attached to the second fastening article.The systems may include two or more wire connecting the first fasteningarticle to the second fastening article. The systems may include aprocessing device attached to the second fastening article Theprocessing device may be a micro-controller. The processing device maybe a microprocessor. The systems may include a processing deviceattached to the first fastening article.

In an eleventh aspect, in general, the subject matter described in thisspecification can be embodied in systems that include an accelerometer,a transmitter connected to the accelerometer and configured to transmitinformation derived from the accelerometer to an external processingdevice, and a ring band connected to the accelerometer that is capableof holding the accelerometer in place on a thumb of a user.

This and other aspects can each optionally include one or more of thefollowing features. The systems may include a gyroscope connected to thering band. The systems may include a processing device attached to thering band. The processing device may be a micro-controller. Theprocessing device may be a microprocessor. The systems may include areceiver configured to receive configuration commands from the externalprocessing device.

In a twelfth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a firstaccelerometer, a first transmitter connected to the first accelerometerand configured to transmit information derived from the firstaccelerometer to an external processing device, a ring band connected tothe first accelerometer that is capable of holding the firstaccelerometer in place on a finger of a user, a second accelerometer; asecond transmitter connected to the second accelerometer and configuredto transmit information derived from the second accelerometer to theexternal processing device, and a wrist band connected to the secondaccelerometer that is capable of holding the second accelerometer inplace on the wrist of a user.

This and other aspects can each optionally include one or more of thefollowing features. The systems may include a magnetometer attached tothe wrist band. The systems may include a magnetometer attached to thering band. The systems may include a gyroscope attached to the wristband. The systems may include a gyroscope attached to the ring band. Thesystems may include a wireless transmitter attached to the wrist band.The systems may include a wireless transmitter attached to the ringband. The systems may include two or more wire connecting the ring bandto the wrist band. The systems may include a processing device attachedto the wrist band. The processing device may be a micro-controller. Theprocessing device may be a microprocessor. The systems may include aprocessing device attached to the ring band.

In a thirteenth aspect, in general, the subject matter described in thisspecification can be embodied in methods that include affixing two ormore sensors to fingers and wrists, wherein signals from the sensors areprocessed to determine locations of movement in two-dimensional orthree-dimension space and to determine locations of taps intwo-dimensional or three-dimensional space.

In a fourteenth aspect, in general, the subject matter described in thisspecification can be embodied in methods that include affixing a firstaccelerometer to the thumb of a user and affixing a second accelerometerto a reference location on the same hand or wrist of the user. Themethods may include processing signals from the first accelerometer todetect taps of the user thumb and processing signals from the first andsecond accelerometers to determine which of a plurality of targetlocations on the hand was tapped by the thumb upon detection of a tap.

This and other aspects can each optionally include one or more of thefollowing features. Signals from the second accelerometer may beprocessed to determine the orientation of the user hand in relation tothe radius of the earth. A value may be selecting among a plurality ofvalues assigned to tap targets based in part on the orientation of thehand to the radius of the earth at the time of the taps. Processing ofsignals from the first and second accelerometers may include trackingthe change in position of the first accelerometer relative to the secondaccelerometer. Processing of signals from the first and secondaccelerometers may include determining an angle between vectorsemanating from the first accelerometer and the second accelerometer.Processing of signals from the first and second accelerometers mayinclude determining a linear displacement of either the first or secondaccelerometers between two time points. The methods may include affixinga first magnetometer to the thumb of a user and affixing a secondmagnetometer to a reference location on the same hand or wrist of theuser. Signals from the first and second magnetometers may be processedto determine which of a plurality of target locations on the hand wastapped by the thumb upon detection of a tap. The methods may includeaffixing a first gyroscope to the thumb of a user and affixing a secondgyroscope to a reference location on the same hand or wrist of the user.Signals from the first and second gyroscopes may be processed todetermine which of a plurality of target locations on the hand wastapped by the thumb upon detection of a tap.

In a fifteenth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a firstaccelerometer and a first fastening article attached to the firstaccelerometer, the first fastening article capable of holding the firstaccelerometer in place on a portion of a thumb of a user. The systemsthat include a second accelerometer and a second fastening articleattached to the second accelerometer, the second fastening articlecapable of holding the second accelerometer in place on a portion of awrist of a user. The systems that include a processing device receivingdata based on acceleration measurements from the first accelerometer andthe second accelerometer via one or more wireless communication links.The systems that include a visual gesture recognition device connectedto the processing device, which is configured to incorporate the datawith image data to identify gestures.

In a sixteenth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a firstaccelerometer and a first fastening article attached to the firstaccelerometer, the first fastening article capable of holding the firstaccelerometer in place on a portion of a thumb of a user. The systemsmay include a second accelerometer and a second fastening articleattached to the second accelerometer, the second fastening articlecapable of holding the second accelerometer in place on a portion of awrist of a user. The systems may include a processing device receivingdata based on acceleration measurements from the first accelerometer andthe second accelerometer via one or more wireless communication links.The systems may include a projector connected to the processing device,the projector configured to project images produced by the processingdevice and wherein the images including graphic representations ofsymbols selected based the data.

In a seventeenth aspect, in general, the subject matter described inthis specification can be embodied in methods that include detecting aninstance of a gesture made with a hand, wherein the gesture isassociated with a plurality of symbols. The methods may includeestimating a first angle between a first axis that is approximatelyparallel to a forearm of the hand and an acceleration experienced by anaccelerometer attached to the forearm. The methods may include assigningone of the plurality of symbols to the instance of the gesture based inpart on the estimate of the first angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between a second axis and theacceleration experienced by the accelerometer attached to the forearmmay be estimated, where the second axis is approximately perpendicularto the first axis. The instance of the gesture may be associated withone of the plurality of symbols based in part on the estimate of thesecond angle.

In an eighteenth aspect, in general, the subject matter described inthis specification can be embodied in systems that include a dataprocessing apparatus and a data storage device storing instructionsexecutable by the data processing apparatus that upon execution by thedata processing apparatus cause the data processing apparatus to performoperations including detecting an instance of a gesture made with ahand, wherein the gesture is associated with a plurality of symbols. Theoperations may include estimating a first angle between a first axisthat is approximately parallel to a forearm of the hand and anacceleration experienced by an accelerometer attached to the forearm.The operations may include assigning one of the plurality of symbols tothe instance of the gesture based in part on the estimate of the firstangle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between a second axis and theacceleration experienced by the accelerometer attached to the forearmmay be estimated, where the second axis is approximately perpendicularto the first axis. The instance of the gesture may be associated withone of the plurality of symbols based in part on the estimate of thesecond angle.

In a nineteenth aspect, in general, the subject matter described in thisspecification can be embodied in systems that include a moduleconfigured to detect an instance of a gesture made with a hand, whereinthe gesture is associated with a plurality of symbols. The systems mayinclude a module configured to estimate a first angle between a firstaxis that is approximately parallel to a forearm of the hand and anacceleration experienced by an accelerometer attached to the forearm.The systems may include a module configured to assign one of theplurality of symbols to the instance of the gesture based in part on theestimate of the first angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between a second axis and theacceleration experienced by the accelerometer attached to the forearmmay be estimated, where the second axis is approximately perpendicularto the first axis. The instance of the gesture may be associated withone of the plurality of symbols based in part on the estimate of thesecond angle.

In a twentieth aspect, in general, the subject matter described in thisspecification can be embodied in one or more computer-readable storagemedia encoded with a computer program. The computer program may includeinstructions that when executed by data a processing apparatus cause thedata processing apparatus to perform operations including detecting aninstance of a gesture made with a hand, wherein the gesture isassociated with a plurality of symbols. The operations may includeestimating a first angle between a first axis that is approximatelyparallel to a forearm of the hand and an acceleration experienced by anaccelerometer attached to the forearm. The operations may includeassigning one of the plurality of symbols to the instance of the gesturebased in part on the estimate of the first angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between a second axis and theacceleration experienced by the accelerometer attached to the forearmmay be estimated, where the second axis is approximately perpendicularto the first axis. The instance of the gesture may be associated withone of the plurality of symbols based in part on the estimate of thesecond angle.

In a twenty-first aspect, in general, the subject matter described inthis specification can be embodied in methods that include detecting aninstance of a gesture made with a hand, wherein the gesture isassociated with a plurality of symbols. The methods may includeestimating an angle between an axis that is approximately perpendicularto a forearm of the hand and an acceleration experienced by anaccelerometer attached to the forearm. The methods may include assigningone of the plurality of symbols to the instance of the gesture based inpart on the estimate of the angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between an acceleration experiencedby an accelerometer attached to the forearm and a second axis that isalso approximately perpendicular to a forearm of the hand may beestimated. The one of the plurality of symbols may be assigned to theinstance of the gesture based in part on the estimate of the secondangle.

In a twenty-second aspect, in general, the subject matter described inthis specification can be embodied in systems that include a dataprocessing apparatus and a data storage device storing instructionsexecutable by the data processing apparatus that upon execution by thedata processing apparatus cause the data processing apparatus to performoperations including detecting an instance of a gesture made with ahand, wherein the gesture is associated with a plurality of symbols. Theoperations may include estimating an angle between an axis that isapproximately perpendicular to a forearm of the hand and an accelerationexperienced by an accelerometer attached to the forearm. The operationsmay include assigning one of the plurality of symbols to the instance ofthe gesture based in part on the estimate of the angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between an acceleration experiencedby an accelerometer attached to the forearm and a second axis that isalso approximately perpendicular to a forearm of the hand may beestimated. The one of the plurality of symbols may be assigned to theinstance of the gesture based in part on the estimate of the secondangle.

In a twenty-third aspect, in general, the subject matter described inthis specification can be embodied in systems that include a moduleconfigured to detect an instance of a gesture made with a hand, whereinthe gesture is associated with a plurality of symbols. The systems mayinclude a module configured to estimate an angle between an axis that isapproximately perpendicular to a forearm of the hand and an accelerationexperienced by an accelerometer attached to the forearm. The systems mayinclude a module configured to assign one of the plurality of symbols tothe instance of the gesture based in part on the estimate of the angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between an acceleration experiencedby an accelerometer attached to the forearm and a second axis that isalso approximately perpendicular to a forearm of the hand may beestimated. The one of the plurality of symbols may be assigned to theinstance of the gesture based in part on the estimate of the secondangle.

In a twenty-fourth aspect, in general, the subject matter described inthis specification can be embodied in one or more computer-readablestorage media encoded with a computer program. The computer program mayinclude instructions that when executed by data a processing apparatuscause the data processing apparatus to perform operations includingdetecting an instance of a gesture made with a hand, wherein the gestureis associated with a plurality of symbols. The operations may includeestimating an angle between an axis that is approximately perpendicularto a forearm of the hand and an acceleration experienced by anaccelerometer attached to the forearm. The operations may includeassigning one of the plurality of symbols to the instance of the gesturebased in part on the estimate of the angle.

This and other aspects can each optionally include one or more of thefollowing features. The acceleration may be caused by gravitationalforce of the Earth. A second angle between an acceleration experiencedby an accelerometer attached to the forearm and a second axis that isalso approximately perpendicular to a forearm of the hand may beestimated. The one of the plurality of symbols may be assigned to theinstance of the gesture based in part on the estimate of the secondangle.

In a twenty-fifth aspect, in general, the subject matter described inthis specification can be embodied in methods that include receiving afirst set of sensor measurements from a first sensor module that isattached to a thumb of a user, wherein the first sensor module comprisesan accelerometer and the first set of sensor measurements includesacceleration measurements. The methods may include receiving a secondset of sensor measurements from a second sensor module that is attachedto a wrist of the user, wherein the second sensor module includes anaccelerometer and the second set of sensor measurements includesacceleration measurements. The methods may include determining, based atleast in part on the first set of measurements and the second set ofmeasurements, an orientation of the thumb relative to the wrist. Themethods may include detecting, based at least in part on accelerationmeasurements in the second set of sensor measurements, when a hand ofthe user is moved. The methods may include, when the orientationsatisfies a condition and movement of the hand is detected, moving acursor in a virtual space in a direction and by an amount determinedbased at least in part on acceleration measurements in the second set ofsensor measurements. The methods may include causing the motion of thecursor in the virtual space to be reflected in a visual representationof the virtual space that is presented on a display.

This and other aspects can each optionally include one or more of thefollowing features. The condition may require the angle between an axisof a sensor in the first sensor module that is approximately parallel toa portion of the thumb and an axis of a sensor in the second sensormodule that is approximately parallel a forearm of the user to be belowa threshold. Tap events may be detected, based on the first set ofsensor measurements and the second set of sensor measurements, when thethumb taps a tap target located on another finger of the hand. Detectionof a thumb tap on the tap target may cause an interaction with an objectin the virtual space that the cursor is in contact with. The secondsensor module may include a gyroscope and the second set of sensormeasurements may include angular rate measurements. A gesture may bedetected based on the first set of sensor measurements and the secondset of sensor measurements, the gesture indicating a selection of anobject in the virtual space that the cursor is in contact with. Theselected object in virtual space may be rotated, based at least in parton the angular rate measurements in the second set of sensormeasurements. The second sensor module may include a magnetometer andthe second set of sensor measurements may include magnetic fluxmeasurements. A gesture may be detected based on the first set of sensormeasurements and the second set of sensor measurements, the gestureindicating a selection of an object in the virtual space that the cursoris in contact with. The object in virtual space may be rotated, based atleast in part on acceleration measurements and magnetic fluxmeasurements in the second set of sensor measurements. The first sensormodule may include a magnetometer and the first set of sensormeasurements may include magnetic flux measurements and the secondsensor module may include a magnetometer and the second set of sensormeasurements may include magnetic flux measurements. Determining theorientation may include determining, based on the first set of sensormeasurements, a first estimate of orientation of the first sensor modulerelative to a gravitational field and a magnetic field; determining,based on the second set of sensor measurements, a second estimate oforientation of the second sensor module relative to a gravitationalfield and a magnetic field; and comparing the first estimate oforientation to the second estimate if orientation. The first sensormodule may include a gyroscope and the first set of sensor measurementsmay include angular rate measurements and the second sensor module mayinclude a gyroscope and the second set of sensor measurements mayinclude angular rate measurements. Determining the orientation mayinclude determining, based on the first set of sensor measurements, afirst estimate of orientation of the first sensor module relative to areference orientation; determining, based on the second set of sensormeasurements, a second estimate of orientation of the second sensormodule relative to a reference orientation; and comparing the firstestimate of orientation to the second estimate if orientation.

In a twenty-sixth aspect, in general, the subject matter described inthis specification can be embodied in systems that include a dataprocessing apparatus and a data storage device storing instructionsexecutable by the data processing apparatus that upon execution by thedata processing apparatus cause the data processing apparatus to performoperations including receiving a first set of sensor measurements from afirst sensor module that is attached to a thumb of a user, wherein thefirst sensor module comprises an accelerometer and the first set ofsensor measurements includes acceleration measurements. The operationsmay include receiving a second set of sensor measurements from a secondsensor module that is attached to a wrist of the user, wherein thesecond sensor module includes an accelerometer and the second set ofsensor measurements includes acceleration measurements. The operationsmay include determining, based at least in part on the first set ofmeasurements and the second set of measurements, an orientation of thethumb relative to the wrist. The operations may include detecting, basedat least in part on acceleration measurements in the second set ofsensor measurements, when a hand of the user is moved. The operationsmay include, when the orientation satisfies a condition and movement ofthe hand is detected, moving a cursor in a virtual space in a directionand by an amount determined based at least in part on accelerationmeasurements in the second set of sensor measurements. The operationsmay include causing the motion of the cursor in the virtual space to bereflected in a visual representation of the virtual space that ispresented on a display.

This and other aspects can each optionally include one or more of thefollowing features. The condition may require the angle between an axisof a sensor in the first sensor module that is approximately parallel toa portion of the thumb and an axis of a sensor in the second sensormodule that is approximately parallel a forearm of the user to be belowa threshold. Tap events may be detected, based on the first set ofsensor measurements and the second set of sensor measurements, when thethumb taps a tap target located on another finger of the hand. Detectionof a thumb tap on the tap target may cause an interaction with an objectin the virtual space that the cursor is in contact with. The secondsensor module may include a gyroscope and the second set of sensormeasurements may include angular rate measurements. A gesture may bedetected based on the first set of sensor measurements and the secondset of sensor measurements, the gesture indicating a selection of anobject in the virtual space that the cursor is in contact with. Theselected object in virtual space may be rotated, based at least in parton the angular rate measurements in the second set of sensormeasurements. The second sensor module may include a magnetometer andthe second set of sensor measurements may include magnetic fluxmeasurements. A gesture may be detected based on the first set of sensormeasurements and the second set of sensor measurements, the gestureindicating a selection of an object in the virtual space that the cursoris in contact with. The object in virtual space may be rotated, based atleast in part on acceleration measurements and magnetic fluxmeasurements in the second set of sensor measurements. The first sensormodule may include a magnetometer and the first set of sensormeasurements may include magnetic flux measurements and the secondsensor module may include a magnetometer and the second set of sensormeasurements may include magnetic flux measurements. Determining theorientation may include determining, based on the first set of sensormeasurements, a first estimate of orientation of the first sensor modulerelative to a gravitational field and a magnetic field; determining,based on the second set of sensor measurements, a second estimate oforientation of the second sensor module relative to a gravitationalfield and a magnetic field; and comparing the first estimate oforientation to the second estimate if orientation. The first sensormodule may include a gyroscope and the first set of sensor measurementsmay include angular rate measurements and the second sensor module mayinclude a gyroscope and the second set of sensor measurements mayinclude angular rate measurements. Determining the orientation mayinclude determining, based on the first set of sensor measurements, afirst estimate of orientation of the first sensor module relative to areference orientation; determining, based on the second set of sensormeasurements, a second estimate of orientation of the second sensormodule relative to a reference orientation; and comparing the firstestimate of orientation to the second estimate if orientation.

In a twenty-seventh aspect, in general, the subject matter described inthis specification can be embodied in systems that include a wirelessnetworking interface configured to receive a first set of sensormeasurements from a first sensor module that is attached to a thumb of auser, wherein the first sensor module includes an accelerometer and thefirst set of sensor measurements includes acceleration measurements. Thesystems may include a wireless networking interface configured toreceive a second set of sensor measurements from a second sensor modulethat is attached to a wrist of the user, wherein the second sensormodule includes an accelerometer and the second set of sensormeasurements includes acceleration measurements. The systems may includea module configured to determine, based at least in part on the firstset of measurements and the second set of measurements, an orientationof the thumb relative to the wrist. The systems may include a moduleconfigured to detect, based at least in part on accelerationmeasurements in the second set of sensor measurements, when a hand ofthe user is moved. The systems may include a module configured to, whenthe orientation satisfies a condition and movement of the hand isdetected, move a cursor in a virtual space in a direction and by anamount determined based at least in part on acceleration measurements inthe second set of sensor measurements. The systems may include a moduleconfigured to cause the motion of the cursor in the virtual space to bereflected in a visual representation of the virtual space that ispresented on a display.

This and other aspects can each optionally include one or more of thefollowing features. The condition may require the angle between an axisof a sensor in the first sensor module that is approximately parallel toa portion of the thumb and an axis of a sensor in the second sensormodule that is approximately parallel a forearm of the user to be belowa threshold. Tap events may be detected, based on the first set ofsensor measurements and the second set of sensor measurements, when thethumb taps a tap target located on another finger of the hand. Detectionof a thumb tap on the tap target may cause an interaction with an objectin the virtual space that the cursor is in contact with. The secondsensor module may include a gyroscope and the second set of sensormeasurements may include angular rate measurements. A gesture may bedetected based on the first set of sensor measurements and the secondset of sensor measurements, the gesture indicating a selection of anobject in the virtual space that the cursor is in contact with. Theselected object in virtual space may be rotated, based at least in parton the angular rate measurements in the second set of sensormeasurements. The second sensor module may include a magnetometer andthe second set of sensor measurements may include magnetic fluxmeasurements. A gesture may be detected based on the first set of sensormeasurements and the second set of sensor measurements, the gestureindicating a selection of an object in the virtual space that the cursoris in contact with. The object in virtual space may be rotated, based atleast in part on acceleration measurements and magnetic fluxmeasurements in the second set of sensor measurements. The first sensormodule may include a magnetometer and the first set of sensormeasurements may include magnetic flux measurements and the secondsensor module may include a magnetometer and the second set of sensormeasurements may include magnetic flux measurements. Determining theorientation may include determining, based on the first set of sensormeasurements, a first estimate of orientation of the first sensor modulerelative to a gravitational field and a magnetic field; determining,based on the second set of sensor measurements, a second estimate oforientation of the second sensor module relative to a gravitationalfield and a magnetic field; and comparing the first estimate oforientation to the second estimate if orientation. The first sensormodule may include a gyroscope and the first set of sensor measurementsmay include angular rate measurements and the second sensor module mayinclude a gyroscope and the second set of sensor measurements mayinclude angular rate measurements. Determining the orientation mayinclude determining, based on the first set of sensor measurements, afirst estimate of orientation of the first sensor module relative to areference orientation; determining, based on the second set of sensormeasurements, a second estimate of orientation of the second sensormodule relative to a reference orientation; and comparing the firstestimate of orientation to the second estimate if orientation.

In a twenty-eighth aspect, in general, the subject matter described inthis specification can be embodied in one or more computer-readablestorage media encoded with a computer program. The computer program mayinclude instructions that when executed by a data processing apparatuscause the data processing apparatus to perform operations includingreceiving a first set of sensor measurements from a first sensor modulethat is attached to a thumb of a user, wherein the first sensor modulecomprises an accelerometer and the first set of sensor measurementsincludes acceleration measurements. The operations may include receivinga second set of sensor measurements from a second sensor module that isattached to a wrist of the user, wherein the second sensor moduleincludes an accelerometer and the second set of sensor measurementsincludes acceleration measurements. The operations may includedetermining, based at least in part on the first set of measurements andthe second set of measurements, an orientation of the thumb relative tothe wrist. The operations may include detecting, based at least in parton acceleration measurements in the second set of sensor measurements,when a hand of the user is moved. The operations may include, when theorientation satisfies a condition and movement of the hand is detected,moving a cursor in a virtual space in a direction and by an amountdetermined based at least in part on acceleration measurements in thesecond set of sensor measurements. The operations may include causingthe motion of the cursor in the virtual space to be reflected in avisual representation of the virtual space that is presented on adisplay.

This and other aspects can each optionally include one or more of thefollowing features. The condition may require the angle between an axisof a sensor in the first sensor module that is approximately parallel toa portion of the thumb and an axis of a sensor in the second sensormodule that is approximately parallel a forearm of the user to be belowa threshold. Tap events may be detected, based on the first set ofsensor measurements and the second set of sensor measurements, when thethumb taps a tap target located on another finger of the hand. Detectionof a thumb tap on the tap target may cause an interaction with an objectin the virtual space that the cursor is in contact with. The secondsensor module may include a gyroscope and the second set of sensormeasurements may include angular rate measurements. A gesture may bedetected based on the first set of sensor measurements and the secondset of sensor measurements, the gesture indicating a selection of anobject in the virtual space that the cursor is in contact with. Theselected object in virtual space may be rotated, based at least in parton the angular rate measurements in the second set of sensormeasurements. The second sensor module may include a magnetometer andthe second set of sensor measurements may include magnetic fluxmeasurements. A gesture may be detected based on the first set of sensormeasurements and the second set of sensor measurements, the gestureindicating a selection of an object in the virtual space that the cursoris in contact with. The object in virtual space may be rotated, based atleast in part on acceleration measurements and magnetic fluxmeasurements in the second set of sensor measurements. The first sensormodule may include a magnetometer and the first set of sensormeasurements may include magnetic flux measurements and the secondsensor module may include a magnetometer and the second set of sensormeasurements may include magnetic flux measurements. Determining theorientation may include determining, based on the first set of sensormeasurements, a first estimate of orientation of the first sensor modulerelative to a gravitational field and a magnetic field; determining,based on the second set of sensor measurements, a second estimate oforientation of the second sensor module relative to a gravitationalfield and a magnetic field; and comparing the first estimate oforientation to the second estimate if orientation. The first sensormodule may include a gyroscope and the first set of sensor measurementsmay include angular rate measurements and the second sensor module mayinclude a gyroscope and the second set of sensor measurements mayinclude angular rate measurements. Determining the orientation mayinclude determining, based on the first set of sensor measurements, afirst estimate of orientation of the first sensor module relative to areference orientation; determining, based on the second set of sensormeasurements, a second estimate of orientation of the second sensormodule relative to a reference orientation; and comparing the firstestimate of orientation to the second estimate if orientation.

In a twenty-ninth aspect, in general, the subject matter described inthis specification can be embodied in systems that include a firstgyroscope; a first fastening article attached to the first gyroscope,the first fastening article capable of holding the first gyroscope inplace on a portion of a thumb of a user; a second gyroscope; and asecond fastening article attached to the second gyroscope, the secondfastening article capable of holding the second gyroscope in place on aportion of a wrist of a user.

This and other aspects can each optionally include one or more of thefollowing features. The first fastening article may be a ring band. Thesecond fastening article may be a wrist band. A system may include aprocessing device configured to receive angular rate measurements fromthe first gyroscope and determine, based at least in part on thereceived angular rate measurements, when a thumb of a user wearing thefirst fastening article and the second fastening article has been tappedagainst a surface. The processing device may be further configured toreceive angular rate measurements from the second gyroscope and, when atap of the thumb is detected, determine, based at least in part onangular rate measurements from the first gyroscope and the secondgyroscope, which tap target from among a set of tap targets located onfingers of a hand of the user have been tapped by the thumb. Theprocessing device may be further configured to map the tap of the taptarget to a symbol associated with the tap target. A system may includetwo or more wires that connect the first fastening device to the secondfastening device. A system may include a wireless transmitter attachedto the second fastening article, and the wireless transmitter may beconfigured to transmit data based on angular rate measurements from thesecond gyroscope to a computing device. A system may include a wirelesstransmitter attached to the second fastening article, and the wirelesstransmitter may be configured to transmit data based on angular ratemeasurements from the first gyroscope to a computing device. A systemmay include a display device configured to present images produced bythe processing device and the images may include graphic representationsof symbols selected based the data. The display device may be a liquidcrystal display. The display device may be a projector. The processingdevice may be attached to the second fastening article. The processingdevice may be part of a remote computing device.

Implementations may include zero or more of the following advantages.Some implementations may reliably detect and classify hand gestures toallow a user to control a computing device. Some implementations mayinclude sensor components that are comfortably wearable on a thumband/or wrist of a user. Some implementations may enable a user to inputalpha-numeric text or other symbols to a computing device. Someimplementations may enable a user to manipulate a cursor in a twodimensional or a three dimensional virtual workspace. Someimplementations may be robust to environmental noise such as vibrationsor accelerations experienced in a moving vehicle. Some implementationsmay enable a user to enter text on mobile device without using limiteddisplay space to present keys. Some implementations may enable a user toenter symbols or commands to a computing device by tapping tap targetswithout looking at those targets.

The details of one or more examples are set forth in the accompanyingdrawings and the description below. Further features, aspects, andadvantages of the disclosed invention will become apparent from thedescription, the drawings, and the claims.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing of an example interface ring.

FIG. 2 is a drawing of an example interface wrist band.

FIG. 3 is a drawing of a hand wearing an example interface ring andwrist band with the thumb pressed to the distal phalanx of the middlefinger.

FIG. 4 is a drawing of a hand with example target locations on thefingers indicated.

FIG. 5 is a drawing of a user wearing an an interface ring and wristband and tapping a target with the forearm perpendicular to the Earthradius.

FIG. 6 is a drawing of a user wearing an an interface ring and wristband and tapping a target with the forearm at 50-degree angle to theEarth radius.

FIG. 7 is a drawing of a user wearing an an interface ring and wristband and tapping a target with the palm facing up.

FIG. 8 is a drawing of a user wearing an an interface ring and wristband and tapping a target with the palm facing sideways.

FIG. 9 is a drawing of an example interface system, which includes acombined processing-and-display unit, an interface ring, and aninterface wrist band.

FIG. 10A-10C are a table illustrating an example mapping of tap-targetand hand-orientation pairs to distinct characters.

FIG. 11 is a flowchart of an example process 1100 for interpretingsignals from a user computing interface

DETAILED DESCRIPTION

Computing interfaces are described for controlling a target computingdevice (e.g., a smart-phone, a tablet device, a laptop computer, atelevision, automobile environmental control system, or some otherdevice that includes a microprocessor and accepts user input). In someimplementations, a computer interface includes a sensor module that isattached to fastening article (e.g., a ring band, an adhesive substrate,or a glove with a thumb sleeve) that is capable of holding the sensormodule in place on a portion of a thumb of a user. The sensor module mayinclude an accelerometer, a magnetometer, and/or a gyroscope. In someimplementations, a computing interface also includes a reference sensormodule that may be attached to fastening article (e.g., a wrist band orsleeve) that is capable of holding the reference sensor module in placeon a portion of the wrist of a user (or some other reference location onthe hand or forearm of the user). A reference sensor module may includean accelerometer, a magnetometer, and/or a gyroscope.

A sensor module of a computing interface may also include amicro-controller or microprocessor, a wireless transmitter, and/or abattery. In some implementations, two sensor modules of an interface maybe connected by two or more wires (e.g., a serial port cable), and onlyone of the sensor modules includes a battery that supplies power to bothsensor modules. In some implementations, each sensor module has its ownbattery and is configured to transmit measurements (e.g., accelerationmeasurements, magnetic flux measurements, and/or angular ratemeasurements) from one or more sensors in the sensor module to a remotecomputing device via a wireless communications link (e.g., a Bluetoothlink). In some implementations, a sensor module transmits (e.g., viawireless communications link) its sensor measurements to another sensormodule in the interface rather than directly to a remote computingdevice. Data based on sensor measurements from multiple sensor modulesmay be transmitted from one of the sensor modules (e.g., a referencesensor module attached to wrist) to a computing device that the userseeks to control or provide input to. For example, measurements from allthe sensors in an interface may be forwarded to the target computingdevice via a transmitter (e.g., a Bluetooth transmitter) included in areference sensor module. In some implementations, one of the sensormodules (a reference sensor module attached to wrist) includes aprocessing device (e.g., a micro-controller or a microprocessor) thatanalyzes sensor measurements from sensors of the interface and transmitsother data based on those measurements to the target computing device.For example, symbols assigned to thumb taps detected by the interfacemay be transmitted to a target computing device.

In some implementations, processing to interpret the measurements fromone or more sensors of an interface is performed by an application ordevice driver that runs on the target computing device.

Example processes are described for interpreting measurements fromsensors in various interface configurations. The example interfaces withcorresponding processes may enable a computing device to determine whena thumb of a user wearing an interface is tapped against a surface. Forexample, a user's thumb may be tapped against one of a set of configuredtap targets on the other fingers of the user. These tap events may bedetected and classified to identify which tap target was tapped and tomap that tap gesture to a corresponding symbol that the user intends toinput to the target computing device. In some implementations, theorientation of a user's wrist may be determined and used to select amongmultiple symbols assigned to an individual tap target.

In some implementations, an interface may support a cursor manipulationmode that enables a user to interact with objects in a virtual space(e.g., a two dimensional or three dimensional virtual space). Forexample, when in cursor manipulation mode, acceleration measurementsfrom an accelerometer in the interface analyzed to control the movementof a cursor in the virtual space. In some implementations, angular ratemeasurements from a gyroscope in an interface may be interpreted toenable a user to rotate objects in virtual space that have been selectedwith a cursor while in a cursor manipulation mode.

Referring to FIG. 1, a computing interface may include a ring 100 thatmay be worn on a user's thumb. The ring includes a band 110 and one ormore accelerometers, magnetometers, or gyroscopes (collectively, a setof sensors) that are located in an electronic component housing 120. Thering may include a single tri-axial or multiple dual-axis and/or singleaxis sensors to span the three dimensional space. In someimplementations, the axes of different types of sensors in the componenthousing 120 may be aligned. In some implementations, the axes ofdifferent types of sensors in the component housing 120 may be alignedelectronically via a calibration process. The ring may also include aradio frequency transmitting device, such as a Bluetooth transmitter,and a battery. The electronic component housing may include a switch 130for powering the electronics components up and down.

The accelerometer may measure the spatial orientation of the thumb andits motion. For example, when the thumb is tapped against a target, suchas phalanges on the other fingers of the hand, an abrupt decelerationresults that is detected by the accelerometer. A transmitter may be usedto send sensor measurements from the ring to an external processingdevice. A transmitter may also be used to send information about eventsderived from sensor measurements to an external processing device.

The ring band 110 serves to hold the interface ring 100 in place on auser's thumb. As thumb size may vary between users, it may beadvantageous to make the ring band flexible enough to comfortably fitthumbs of different sizes. The ring band may be made of plastic, oranother flexible material. The ring band may be of approximatelycircular shape with a single gap 140 that allows the ring band to flexto surround most of the circumference of a user's thumb. The ring bandmay alternatively be formed into a complete loop that completelyencircles the user's thumb when worn. In this case the ring band may bemade of a material, such as nylon, that is capable of stretching in alongitudinal direction. Alternatively the ring band may be rigid andfitted to a particular thumb size.

In some implementations, a ring 100 may also include a wireless receiver(e.g., a Bluetooth receiver) for receiving information from a targetcomputing device or an intermediary device. For example, the ring mayreceive configuration commands from target computing device that setoperating parameters of the ring, such as a usage mode (e.g., to enter acursor control mode), a power-saving mode, a sampling rate for one ormore sensors, etc.

In some implementations, a portion of the ring that includes a Bluetoothtransmitter may be detachable from the band. This portion may alsoinclude a speaker and microphone that allow the detachable component tobe used as a Bluetooth enabled earbud for a cellphone.

Referring to FIG. 2, an example computing interface may include abracelet 200 that may be worn on a user's wrist. The bracelet includes awristband 210 and one or more accelerometers, magnetometers, orgyroscopes (collectively, a set of sensors) that are located in anelectronic component housing 220. The bracelet may include a singletri-axial or multiple dual-axis and/or single axis sensors to span thethree dimensional space. In some implementations, the axes of differenttypes of sensors in the component housing 220 may be aligned. In someimplementations, the axes of different types of sensors in the componenthousing 220 may be aligned electronically via a calibration process. Thebracelet may also include a radio frequency transmitting device, such asa Bluetooth transmitter, and a battery. The accelerometer may measurespatial orientation of the wrist and its motion. A transmitter may beused to send sensor measurements from the bracelet to an externalprocessing device. A transmitter may also be used to send informationabout events derived from the sensor measurements to an externalprocessing device. The electronic component housing may include a switch230 for powering the electronics components up and down.

The wristband 210 serves to hold component(s) of the interface in placeon a user's wrist. As wrist size may vary between users, it may beadvantageous to make the wristband flexible enough to comfortably fitwrists of different sizes. The wristband may be made a flexiblematerial, such as rubber, nylon, or plastic. The wristband may includean adjustable fastening device, such a Velcro strip, snaps, cable tie,or a buckle. The wristband may be formed into a complete loop thatcompletely encircles the user's wrist when worn. Alternatively, thewristband may be a continuous loop made of a material, such as rubber ornylon that is capable of stretching in a longitudinal direction to allowthe band to slide over the hand of the user and still fit the wristtight enough to hold an accelerometer in place on the wrist.

In some implementations, a bracelet 200 may also include a wirelessreceiver (e.g., a Bluetooth receiver) for receiving information from atarget computing device or an intermediary device. For example, the ringmay receive configuration commands from target computing device that setoperating parameters of the ring, such as a usage mode (e.g., to enter acursor control mode), a power-saving mode, a sampling rate for one ormore sensors, etc.

Referring to FIG. 3, an interface 300 may include multiple componentsworn on different parts of a user's hands or arms. In the exampledepicted in FIG. 3, the interface includes a ring 100 and a bracelet 200worn on the same hand 310. Once the interface is in place on a user'swrist and thumb, a position tracker module may be initialized. In someimplementations, the position of the sensors in both the ring and thebracelet are tracked by integrating the dynamic motion detected by bothcomponents. The change in position experienced by the bracelet serves asreference for determining how the position of the thumb has changed inrelationship to the rest of the hand. In this manner the effects ofunrelated user movement, such as turning, sitting, standing, walking orriding in a vehicle, on the position of the thumb may be controlled forto isolate changes in the position of the thumb relative to the rest ofthe hand.

In some implementations, accelerometer readings are sampled at afrequency of about 1 kHz and the resulting digital signals are processedto detect when thumb taps occur and classify taps according to the taptargets that are hit. All or part of the processing may be performed bya microprocessor located on the ring or the bracelet. All or part of theprocessing of accelerometer readings may be performed on a dataprocessing device that receives readings via a radio frequencytransmission from the transmitters on the ring and/or on the bracelet.The data processing device may be an internal or external processingdevice, such as a cellphone, that runs software configured to receivesensor readings or information (e.g., filtered signals and/or symbols)based on those readings from the interface. Alternatively the processingdevice may be a stand-alone device configured to receive informationbased on sensor readings from the interface via the radio frequencytransmission. The stand-alone processing device may in turn passinformation such as detected target tap events to an external processingdevice, such as a computer, via another interface, such as a USB(Universal Serial Bus) port.

In some implementations, the devices in the interface system maycommunicate with each other by means of radio frequency. For example,low-power wireless transmission from a ring with a short range (e.g., 1foot range) may be used to convey measurement from the sensors of thering to a processing device attached to a bracelet, which may in turninterpret those measurements and/or forward they to a target computingdevice via a higher power wireless communications link (e.g., aBluetooth link). Such a configuration may allow a smaller battery to beincluded in the ring than in the bracelet. In some implementations, theymay communicate with each other through wired connections. For example,the ring 100 and bracelet 200 may communicate sensor readings throughwired connections to determine their individual spatial orientations. Asanother example, the bracelet may hold an energy storage that suppliespower to the ring through the wire connections.

Accelerometers in the wristband may also be used to detect the spatialorientation of the hand by measuring the static acceleration caused bythe gravitational pull of the Earth, which is a vector along the Earthradius, extending from the Earth center through the wearer of theinterface. The orientation of the accelerometer to the user's wrist maybe fixed by the wristband. Thus, the axes of the three dimensions sensedby the accelerometers may be fixed with respect to the orientation ofthe user's wrist. The angle of the Earth-radius vector with respect tothe reference frame extended by the axes is calculated to determine theorientation of the wrist with respect to the Earth radius. Similarly,the angle of the Earth-radius vector with respect to the reference frameextended by the axes of the ring accelerometers is calculated todetermine the angle of a phalanx of the thumb with respect to the Earthradius.

The Earth-radius angles of the thumb and wrist may be compared toestimate a component of the angle between the thumb and the wristoutside of the plane orthogonal to the Earth radius. The angle betweenthe thumb and the wrist at the time that a tap is detected may be usedto distinguish tap targets on the hand. Information about the currentangle between the thumb and the wrist may be used in conjunction withinformation from a position tracking module to classify tap events byassigning them to a tap target.

In some implementations, magnetometers may be used in conjunction withaccelerometers to to determine the relative orientations of the thumbring and a reference device, such as a reference device located on thewrist. The thumb ring may include a tri-axial accelerometer and atri-axial magnetometer. The axes of these magnetometer and theaccelerometer may be aligned. The reference device may also include anaccelerometer and a magnetometer whose axes are aligned. When a tapevent is detected, readings from the thumb sensors and the referencessensors are windowed and sampled to estimate the acceleration andmagnetic field experienced by the thumb sensors and the referencesensors while the thumb is at rest relative to the rest of the handduring a tap event. These estimates are encoded as four 3-dimensionalvectors: {right arrow over (t)}_(a), {right arrow over (t)}_(m), {rightarrow over (r)}_(a), {right arrow over (r)}_(m). For the ensuingdisclosure, we use the following notations:

{right arrow over (r)}_(m): The magnetic flux vector detected by themagnetometer in the reference device.

{right arrow over (r)}_(a): The acceleration vector detected by theaccelerometer in the reference device.

{right arrow over (t)}_(m): The magnetic flux vector detected by themagnetometer in the thumb device.

{right arrow over (t)}_(a): The acceleration vector detected by theaccelerometer in the thumb device.

_(x){right arrow over (t)}: The x component of the vector {right arrowover (t)} and similarly for the y and z components.

R_(m): The rotation matrix for aligning the magnetic flux vectors.

R_(a): The rotation matrix for aligning the acceleration vectors.

R: The rotation matrix for aligning both the magnetic flux andacceleration vectors.

A rotation that represents the relative orientation of the thumb deviceand the reference device during the tap event may be determined fromthose four vectors. That rotation may be determined in stages by firstdetermining two component rotations, R_(m) and R_(a), and then combiningthem. First a rotation that aligns the two magnetic field vectors iscalculated by taking a cross product of {right arrow over (r)}_(m) and{right arrow over (t)}_(m) to determine the axis of a minimum-anglerotation that aligns the two vectors as well as the magnitude of theangle of rotation. A dot product is also calculated to disambiguate thequadrant of the angle. These calculations yield an axis/anglerepresentation of the first component rotation.

$\begin{matrix}{{{Cross}\mspace{14mu}{product}\mspace{14mu}{of}\mspace{14mu}{magnetic}\mspace{14mu}{field}\mspace{14mu}{vectors}}{{\overset{->}{x}}_{m} = \frac{{\overset{->}{t}}_{m} \times {\overset{->}{r}}_{m}}{{{\overset{->}{t}}_{m}}{{\overset{->}{r}}_{m}}}}} & {{Eq}\mspace{14mu} 1} \\{{{The}\mspace{14mu}{sine}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{angle}\mspace{14mu}{of}\mspace{14mu}{rotation}}{s_{m} = {{\sin\left( \theta_{m} \right)} = {{\overset{->}{x}}_{m}}}}} & {{Eq}\mspace{14mu} 2} \\{{{The}\mspace{14mu}{cosine}\mspace{14mu}{of}\mspace{11mu}{the}\mspace{14mu}{angle}\mspace{14mu}{of}\mspace{14mu}{rotation}}{c_{m} = {{\cos\left( \theta_{m} \right)} = \frac{{\overset{->}{t}}_{m} \cdot {\overset{->}{r}}_{m}}{{{\overset{->}{t}}_{m}}{{\overset{->}{r}}_{m}}}}}} & {{Eq}\mspace{14mu} 3} \\{{{The}\mspace{14mu}{normalized}\mspace{14mu}{axis}\mspace{14mu}{of}\mspace{14mu}{rotation}}{{\overset{->}{n}}_{m} = {\frac{{\overset{->}{x}}_{m}}{{\overset{->}{x}}_{m}} = \frac{{\overset{->}{x}}_{m}}{\sin\left( \theta_{m} \right)}}}} & {{Eq}\mspace{14mu} 4}\end{matrix}$

The first component rotation can be computed from the rotation axis andthe rotation angle. Let c _(m)=1−c_(m). The first component rotation isrepresented by a 3×3 matrix, R_(m):

$\begin{matrix}{\mspace{79mu}{{{Rotation}\mspace{14mu}{matrix}\mspace{14mu}{for}\mspace{14mu}{magnet}\mspace{14mu}{field}\mspace{14mu}{vectors}}{R_{m} = {\quad\left\lbrack \begin{matrix}{{{\overset{->}{\,_{x}n}}_{m}^{2}{\overset{\_}{c}}_{m}} + c_{m}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} - {{{}_{}^{}{n->}_{}^{}}s_{m}}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} + {{{}_{}^{}{n->}_{}^{}}s_{m}}} \\{{{\overset{->}{\,{\,_{y}n}}}_{m}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} + {{{}_{}^{}{n->}_{}^{}}s_{m}}} & {{{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} + c_{m}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} - {{{}_{}^{}{n->}_{}^{}}s_{m}}} \\{{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} - {{{}_{}^{}{}_{}^{}}s_{m}}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} + {{{}_{}^{}{n->}_{}^{}}s_{m}}} & {{{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{m}} + c_{m}}\end{matrix} \right\rbrack}}}} & {{Eq}\mspace{14mu} 5}\end{matrix}$

The first rotation matrix is then applied to the thumb accelerationvector, {right arrow over (t)}_(a), to determined the rotated thumbacceleration vector, {right arrow over (t)}_(a)′.{right arrow over (t)}_(a)′=R_(m){right arrow over (t)}_(a)  Eq 6:Rotated thumb acceleration vector

A second component rotation that aligns {right arrow over (t)}_(a)′ withthe reference acceleration, {right arrow over (r)}_(a), may bedetermined next. The second component rotation may be constrained to usean axis of rotation aligned with the reference magnetic field, {rightarrow over (r)}_(m), so that alignment of the two magnetic field vectorsis preserved by the second component rotation. That can be done, forexample, using the projections of {right arrow over (r)}_(a) and {rightarrow over (t)}_(a)′ onto the plane perpendicular to {right arrow over(r)}_(m).

$\begin{matrix}{{{Projection}\mspace{14mu}{of}\mspace{14mu}{reference}\mspace{14mu}{acceleration}\mspace{14mu}{vector}}{{\overset{->}{p}}_{r} = {{\overset{->}{r}}_{a} - {\left( {{\overset{->}{r}}_{a} \cdot \frac{{\overset{->}{r}}_{m}}{{\overset{->}{r}}_{m}}} \right)\frac{{\overset{->}{r}}_{m}}{{\overset{->}{r}}_{m}}}}}} & {{Eq}\mspace{14mu} 7} \\{{{Normalized}\mspace{14mu}{projection}\mspace{14mu}{vector}}{{\overset{->}{p}}_{r}^{n} = \frac{{\overset{->}{p}}_{r}}{{\overset{->}{p}}_{r}}}} & {{Eq}\mspace{14mu} 8} \\{{{Projection}\mspace{14mu}{of}\mspace{14mu}{thumb}\mspace{14mu}{acceleration}\mspace{14mu}{vector}}{{\overset{->}{p}}_{t} = {{\overset{->}{t}}_{a} - {\left( {{\overset{->}{t}}_{a} \cdot \frac{{\overset{->}{r}}_{m}}{{\overset{->}{r}}_{m}}} \right)\frac{{\overset{->}{r}}_{m}}{{\overset{->}{r}}_{m}}}}}} & {{Eq}\mspace{14mu} 9} \\{{{Normalized}\mspace{14mu}{projection}\mspace{14mu}{vector}}{{\overset{->}{p}}_{t}^{n} = \frac{{\overset{->}{p}}_{t}}{{\overset{->}{p}}_{t}}}} & {{Eq}\mspace{14mu} 10}\end{matrix}$

The minimum angle rotation to align these projections will then have anaxis of rotation parallel to {right arrow over (r)}_(m), so a crossproduct and dot product may be applied to the projections to determinethe angle of rotation in this plane that will align the projections ofthe acceleration vectors. This second component rotation may also becomputed from an axis/angle representation and may be represented as amatrix, R_(a).

$\begin{matrix}{\mspace{79mu}{{{Cross}\mspace{14mu}{product}\mspace{14mu}{of}\mspace{14mu}{projected}\mspace{14mu}{acceleration}\mspace{14mu}{vectors}}\mspace{79mu}{{\overset{->}{x}}_{a} = {{\overset{->}{p}}_{t}^{n} \times {\overset{->}{p}}_{r}^{n}}}}} & {{Eq}\mspace{14mu} 11} \\{\mspace{79mu}{{{The}\mspace{14mu}{sine}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{angle}\mspace{14mu}{of}\mspace{14mu}{rotation}}\mspace{79mu}{s_{a} = {{\sin\left( \theta_{a} \right)} = {{\overset{->}{x}}_{a}}}}}} & {{Eq}\mspace{14mu} 12} \\{\mspace{79mu}{{{The}\mspace{14mu}{cosine}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{angle}\mspace{14mu}{of}\mspace{14mu}{rotation}}\mspace{79mu}{c_{a} = {{\cos\left( \theta_{a} \right)} = {{\overset{->}{p}}_{t}^{n} \cdot {\overset{->}{p}}_{r}^{n}}}}}} & {{Eq}\mspace{14mu} 13} \\{\mspace{79mu}{{{The}\mspace{14mu}{cosine}\mspace{14mu}{of}\mspace{14mu}{the}\mspace{14mu}{angle}\mspace{14mu}{of}\mspace{14mu}{rotation}}\mspace{79mu}{c_{a} = {{\cos\left( \theta_{a} \right)} = {{\overset{->}{p}}_{t}^{n} \cdot {\overset{->}{p}}_{r}^{n}}}}}} & {{Eq}\mspace{14mu} 14} \\{\mspace{79mu}{{{Rotation}\mspace{14mu}{matrix}\mspace{14mu}{for}\mspace{14mu}{magnet}\mspace{14mu}{field}\mspace{14mu}{vectors}}{R_{a} = {\quad\begin{bmatrix}{{{\overset{->}{\,_{x}n}}_{a}^{2}{\overset{\_}{c}}_{a}} + c_{a}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} - {{{}_{}^{}{n->}_{}^{}}s_{a}}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} + {{{}_{}^{}{n->}_{}^{}}s_{a}}} \\{{{\overset{->}{\,{\,_{y}n}}}_{a}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} + {{{}_{}^{}{n->}_{}^{}}s_{a}}} & {{{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} + c_{a}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} - {{{}_{}^{}{n->}_{}^{}}s_{a}}} \\{{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} - {{{}_{}^{}{}_{}^{}}s_{a}}} & {{{{}_{}^{}{n->}_{}^{}}{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} + {{{}_{}^{}{n->}_{}^{}}s_{a}}} & {{{{}_{}^{}{n->}_{}^{}}{\overset{\_}{c}}_{a}} + c_{a}}\end{bmatrix}}}}} & {{Eq}\mspace{14mu} 15}\end{matrix}$

The two component rotations may then be combined by multiplying the twomatrices in the proper order to produce a matrix representation of therelative orientation of the two devices, R.R=R_(a)R_(m)  Eq 16: Complete rotation matrix

The relative orientation of the thumb and reference devices may beconverted from the matrix representation to a lower dimensionalrepresentation to enable more efficient slicing to quantize theorientation estimate into a symbol estimate. For example, the matrixrepresentation, R, may be converted to an axis/angle representationusing an eigenvalue decomposition. Since the axis of the rotation is aunit vector, the axis/angle may be expressed as a three-tuple bymultiplying the axis by the angle of rotation. These three-tuples maythen be assigned to symbol estimates by slicing in the three dimensionalspace.

A slicer for the orientation estimates may be generated using standardtechniques applied to corpus of tap orientation measurements takenduring a known tap sequence. For example the centroids of clusters oforientation estimates corresponding to a particular tap may be used.Slicing may be accomplished by determining the nearest tap centroid to anew orientation estimate. Slicer regions may be determined based onaggregated data for many users or for a particular user by usingtraining sequences to collect data from that user. In some cases anabbreviated training sequence may be used to customize generic slicerregions to a particular user.

The order of the decomposition of the orientation rotation intocomponents may be reversed. For example, the component required to alignthe acceleration vectors may be determined first and then a constrainedcomponent rotation to approximately align the magnetic field vectors maybe subsequently determined and then combined. The selection of the orderin which the decomposition is performed may be informed by thesignal-to-noise ratios (SNRs) experienced by the accelerometers and themagnetometers.

A calibration sequence may be performed by a user before the first useof the interface. The user may be prompted to execute each step of thecalibration process using a display connected to a processing devicethat the interface is inputting data to. The prompt instructs the userto touch one or more of the targets on the hand and data is recorded astaps are detected. The data may be used only for the current session, orstored in memory as a user profile. In this manner, the interface may betrained to respond to the geometry of the hand and tendencies of aparticular user.

In some implementations ring 100 may include a thermometer that is usedto dynamically adjust an output amplifier gain for one or more of thesensors (e.g., an accelerometer) that have a response characteristicsthat vary with temperature.

In some implementations, not depicted, one or more accelerometerslocated in a second ring worn on one of the proximal phalanges of theother fingers on the hand may be used as a reference for determiningposition and angles of the thumb in relation to the rest of the hand.

Referring to FIG. 4, in some implementations, tap targets are located onphalanges of fingers of a user's hand that the user may ergonomicallytap with the thumb while wearing an example interface including ring 100and bracelet 200. An example layout 400 of tap targets on the hand isshown in FIG. 4. For each of the four other fingers on the hand, the taptargets are centered on the inside surface of the distal 410, middle420, and proximal 430 phalanges. A mapping is established that assignsdifferent symbols to taps of each of the targets. In the figure, eachtarget is labeled by an associated symbol. In some implementations (notshown), the tap targets on an index finger may be centered on the sideof the index finger closest to the thumb. Locating tap targets on thefingers of the user may allow a user to conduct taps without looking atthe tap targets.

An interface may include a matching set of components for the other handso that both hands may be used for data entry. In this case differentsymbols may be assigned to the corresponding tap targets on each hand todouble the size of the symbol set. Furthermore, taps on both hands maybe combined to expand the symbol set even more. For example, tapping andholding the distal phalanx of the left index finger while tapping thephalanges on the other hand may be mapped to one set of symbols; tappingand holding the medial phalanx of the left index finger while tappingthe phalanges on the other hand may be mapped to another set of symbols.In this way, at least 144 (12 phalanges on the left hand times 12phalanges on the right hand) symbols may be produced from combining thetaps on the two hands.

Referring to FIG. 5 and FIG. 6, the angle of the wrist to the Earthradius may be used to distinguish multiple symbols assigned to a singletap target. In example usage scenarios 500 and 600, one of the threeaxes, the z-axis 560, of the accelerometers in the bracelet 200 isapproximately parallel to the forearm of the user 510 and the other twoaxes are labeled x and y. The angle 565 of the z-axis to the Earthradius 550 may be determined and compared to thresholds to distinguishmultiple sets of symbols assigned to the targets on a hand. FIG. 5 showsa user 510 wearing an interface including a ring 100 and a bracelet 200with the forearm oriented at approximately ninety degrees to the Earthradius 550. In this position, the user 510 is able to input one set ofsymbols by tapping the targets on the hand with the thumb. The user 510may access other symbols in scenario 600 by bending the elbow to, forexample, raise the forearm to an angle 665 of approximately fiftydegrees to the Earth Radius 550, as depicted in FIG. 6. In thisposition, the user 510 may input a second set of symbols by tapping thesame targets on the hand. In this manner the multiple symbol sets may beassigned to different ranges of the angle between the user's forearm andthe Earth radius.

Referring to FIG. 7 and FIG. 8, more sets of symbols may bedistinguished by detecting rotations of the wrist. For example, in usagescenario 700, one position for data entry may be with the user's wristturned so the palm of the hand faces up, as depicted by the direction ofthe dashed arrow 760 in FIG. 7. An axis (e.g., an x-axis) of anaccelerometer in bracelet 200 may be approximately parallel to line 760which is perpendicular to the surface of the palm of user 710. The angleof this axis and/or the angle of a second axis that is alsoapproximately orthogonal to a line parallel to the length of the user'sforearm with an acceleration experienced by the accelerometer during atap may be determined and used estimate the orientation of the user'swrist with respect to the Earth Radius 750. In this manner it can bedetermined that the palm of user 710 is facing up and a certain set ofsymbols may be assigned to tap targets on the hand. Usage scenario 800shows another data entry position, in which the wrist is rotated so thatthe palm faces to the side, as depicted in FIG. 8. The dark circles 860illustrate an axis (e.g., an x-axis) of an accelerometer in the bracelet200 that points out of the page. In this scenario 800, the axis 860 thatis approximately perpendicular to the surface of the palm is alsoperpendicular to the Earth Radius 750, while another axis that is alsoapproximately perpendicular to the length of the user's forearm isapproximately parallel with the Earth Radius 750. As long as the forearmis not parallel to the Earth radius, these wrist rotation positions maybe distinguished by comparing the angle between the x or y axes of theaccelerometers in the bracelet and the Earth radius to thresholds. Inthis manner the number of wrist twist positions distinguished canfurther multiply the number of symbols that may be signaled with theinterface.

In some implementations, wrist twists may be detected by tracking fastchanges in position of accelerometers of an interface. For example, thethumb-up and thumb-down gestures, may be detected for signaling approval(OK) and disapproval (CANCEL), respectively, selections in computer userinterface and interaction (UI and UX).

Wrist orientations may also be used to enter different input modes foran interface. For example, turning the wrist so that the palm faces downcould be used to enter a cursor control mode. In cursor control mode,the hand may be moved in the three dimensional space in front of theuser to control a cursor in one or more dimensions. Thumb orientation inrelation to the reference frame on the wrist may be used to determinewhether the cursor is engaged or not, so that the cursor can continue tobe moved in a direction beyond the reach of the user, much like a usermay pick up a mouse or disengage the finger from a trackball. Forexample, the cursor may be disengaged when the thumb is orientedapproximately perpendicular to the length of the forearm of the user(e.g., held in a thumb-up gesture) and the cursor may be engaged whenthe thumb is closer to parallel with the length of the forearm. Forexample, an angle between an axis of a sensor attached to the thumb thatis approximately parallel to a portion of the thumb and an axis of asensor attached to the wrist that is approximately parallel to theforearm may be estimated to determine whether the cursor is engaged. Taptargets may be assigned different symbols in such a mode. For example, atap target may be tapped to select or deselect item highlighted by acursor.

In implementations of an interface that use two thumb rings, certain taptargets may be assigned meta symbols (e.g., ‘shift’ or ‘ctrl’) thatchange the interpretation of target taps by one thumb while the otherthumb is held in place on the meta symbol target.

Referring to FIG. 9, user input received through an interface may beprocessed and/or used by a computing device in a variety of ways. Oneway, is to present graphical representations of symbols indicated by theuser gestures (e.g., thumb taps) made while wearing an interface. FIG. 9illustrates an example interface system in which a user wearing aninterface including a ring 100 and a bracelet 200 performs a thumb tapto cause a combined processing and display device 910 (e.g., a Bluetoothenabled internet television) to display an alpha-numeric characterassociated with the thumb tap gesture. It should be noted that this isjust one of many possible computing devices that may be controlled usingthis type of computing interface. For example, an interface (e.g.,including ring 100 and bracelet 200) could be used to input data to asmartphone with an touchscreen display, a tablet device with atouchscreen display, a computing device controlling a projector, acomputing device controlling other actuators (e.g., an environmentalcontrol system in an automobile), among many other computing devices.

Referring to FIGS. 10A-10C, by combining some of these methods ofdistinguishing symbols by detecting taps of targets on the hand and theconcurrent orientation of the hand, an interface may be used todistinguish a large number of symbols. An example table that maps taptargets and hand orientations to symbols is depicted in FIGS. 10A-10C.

The mapping of tap targets to symbols may be memorized by the user. Asneeded, the mapping of symbols to targets may be depicted on the hand bywearing a thin glove with the symbols drawn on positions associated withtheir targets. The mapping may also be displayed to the user byillustrating the symbols on their tap targets on the image of a hand ona display controlled by the external processing device that data isbeing entered into. The image of the hand(s) with marked targets may besemi-transparent, overlaying the user interface of the underlyingapplication. Such a display could be enabled, disabled or minimized whennot needed by entering a special “help” symbol.

The mapping of targets to symbols may be designed by analyzing therelative frequency of symbols used. For example statistical analysis ofa collection of texts may be conducted to determine which letters andletter sequences are most commonly used. The most commonly occurringsymbols may then be mapped to targets located close together and in thepositions that are most easily accessible to the thumbs. Commonsequences of symbols may have all their symbols assigned to targets thatmay be tapped in quick succession. In this manner, an interface may beoptimized for particular languages or applications. The mapping of taptargets to events may be custom-configured by the users.

FIG. 11 is a flowchart of an example process 1100 for interpretingsignals from a user computing interface. In some implementations, theprocess 1100 may be performed by executing driver software for thecomputing interface on a computing device (e.g., a smart-phone, a tabletdevice, laptop computer, automobile environmental control system, or atelevision) that a user seeks to control by making hand gestures whilewearing the computing interface. A computing device may include amicroprocessor and a data storage device (e.g., flash memory) storinginstructions for causing the computing device to perform process 1100.In some implementations, the process 1100 may be performed by a dataprocessing device (e.g., a micro-controller or microprocessor) attachedto the wrist of a user and symbols derived from the signal processingmay be transmitted to a computing device that the user seeks to control.In some implementations, a data storage device (e.g., flash memory)storing instructions that cause a data processing device to performprocess 1100 may also be attached to the wrist.

The process 1100 may begin by receiving 1110 measurements from sensorsof the interface. The measurements may include acceleration measurementsfrom an accelerometer that is attached to a thumb of a user. In someimplementations, the measurements also include acceleration measurementsfrom a second accelerometer that is attached to a wrist of the user. Insome implementations, the measurements also include magnetic fluxmeasurements from a magnetometer that is attached to the thumb of theuser and magnetic flux measurements from a magnetometer that is attachedto the wrist of the user. In some implementations, the measurements alsoinclude angular rate measurements from a gyroscope that is attached tothe thumb of the user and angular rate measurements from a gyroscopethat is attached to the wrist of the user.

The measurements from the sensors may be received as a time series ofsamples (e.g., sampled at 250 Hz, 500 Hz, 1 KHz, or 2 KHz) from eachsensor output. For example, one or more sensors may be sampled using aco-located micro-controller and resulting samples may be transmittedthrough one or more communications links (e.g., Bluetooth wireless linksand/or a serial port link) to processing device for further analysis. Insome implementations, the time series of samples for each sensor is timesynchronized with the time series of samples for a reference sensor(e.g., the accelerometer attached to the thumb or to the accelerometerattached to the wrist may dictate timing for the other sensor signals).For example, a phase locked loop (PLL) may be implemented to compensatefor clock skew and maintain synchronization with a reference signal froma sensor that is sampled with different clock. A processing devicereceiving the measurements from the sensors may operate as a master in amaster-slave configuration to enforce a sample timing for measurementsreceived from multiple sensor modules of the interface. In someimplementations, a training sequence that causes simultaneousexcitations in two sensors may be used to establish an initial phasesynchronization between the signals from the two sensors. For example,an arm on which accelerometers are worn on the thumb and the wrist maybe swung at the shoulder joint to induce an approximately simultaneouschange in quantities measured by sensors at both locations on the arm.In some implementations, the measurements from two different sensors ofthe interface may be received asynchronously. For example, sensors wornon the right and left hands may be received asynchronously.

In some implementations, the measurements may have been filtered orotherwise processed prior to receiving 1110 the measurements for thesensors. For example, a sequence samples of measurements from anaccelerometer and a co-located gyroscope may be filtered and/orconverted to a sequence of measurements of an orientation (e.g., encodedas Euler angles or a quaternion representation) of the co-locatedsensors by a co-located micro-controller before the measurements arereceived 1110 by an external processing device.

For example, the measurements from the sensors of the interface may bereceived 1110 through a wireless network interface (e.g., a Bluetoothinterface) of a processing device that will interpret the measurements.In some implementations, the measurements may be received by aprocessing device that is co-located with some of the sensors of theinterface (e.g., attached to the wrist of the user). In theseimplementations, the measurements from co-located sensors may bereceived 1110 through a bus or other short range data transfer channelwhile measurements from sensors located further from the processingdevice may be received 1110 through a wireless communication channel(e.g., a Bluetooth link) or though a two or more wires (e.g., a serialport cable) connecting sensor modules of the interface.

A tap of the thumb on a surface may be detected 1120 as an event basedon the received sensor measurements. In some implementations, a tapevent may be detected by filtering a sequence of accelerationmeasurements and/or angular rate measurements from an accelerometerand/or a gyroscope attached to the thumb of the user. Large fast changesin these measurements may be associated with a tap event. For example,the difference between consecutive samples of these measurements may becompared to a threshold (e.g., 1.2 times the acceleration due to Earth'sgravity for the linear acceleration) and when the threshold is exceededa tap event may be detected 1120. A tap detection module may alsoimplement debouncing logic to ignore fast changes in these measurementsfor a short configurable period of time (e.g. 10 or 20 milliseconds)after a tap is detected. For example, tap events may be detected 1120 bya tap detection module of device driver running on a computing device.In some implementations, tap events may be detected 1120 by a tapdetection module running on a processing device that is attached to thewrist of the user.

An orientation of the thumb during the tap event may be determined 1130.When a tap of the thumb is detected, signals (e.g., sequences ofmeasurements) from sensors of the interface may be windowed and/orotherwise filtered in the neighborhood of the detected tap event toestimate characteristics of the tap event. For example, a window ofsensor measurements (e.g., a 5, 10, or 20 millisecond long window) justafter a large deceleration associated with the onset of a tap event maybe averaged to estimate characteristics of the tap event during a briefperiod while the thumb is at rest (relative to the rest of the hand) andin contact with a tap target. In some implementations, the decelerationassociated with the impact is itself considered as a characteristic ofthe tap event. For example, acceleration measurements from anaccelerometer attached to the thumb may be filtered to determine anestimate of a deceleration vector caused by the impact of the thumb withthe tap target. The orientation of the estimated deceleration vectorrelative to the axes of one or more sensors attached to the thumb may bea characteristic considered for classification of a tap event.

In some implementations, an orientation of one or more sensors attachedto the thumb of the user is determined 1130 relative to the anorientation of one or more sensors attached to the wrist of the user.For example, an estimate of the acceleration experienced by anaccelerometer attached to the thumb while the thumb was at rest on thetap target (e.g., acceleration due to the Earth's gravitational force,the acceleration of a vehicle the user is riding in, and/or otherexogenous forces) may be compared to an estimate of the accelerationexperienced by an accelerometer attached to the wrist of the user duringthe same period of time (e.g., a time window just after the decelerationmarking the start of the tap event) to compare the relative orientationsof these accelerations as experienced at each location on the hand orarm of the user. These estimates of acceleration may be determined basedin part on acceleration measurements from the respective accelerometersattached to the thumb and the wrist or some other reference location.

In some implementations, the relative orientation of the thumb and thewrist is determined 1130 based in part on magnetic flux measurementsfrom a magnetometer attached to the thumb and magnetic flux measurementsfrom a magnetometer attached to the wrist. For example, an estimate ofthe magnetic flux experienced by the magnetometer attached to the thumbwhile the thumb was at rest on the tap target (e.g., due to the Earth'smagnetic field, magnetic field from a nearby transformer or power line,and/or other sources of magnetic flux) may be compared to an estimate ofthe magnetic flux experienced by the magnetometer attached to the wristof the user during the same period of time (e.g., a time window justafter the deceleration marking the start of the tap event) to comparethe relative orientations of these magnetic flux vectors as experiencedat each location on the hand or arm of the user. Where the magnetic fluxis approximately uniform in the region of space around the thumb andwrist locations, the orientations of the magnetic flux vectors, asexperienced by the respective magnetometers, may provide informationabout the relative orientation of the two sensors.

In some implementations, an orientation of the thumb relative to thewrist may be determined 1130 by combining information about theacceleration and magnetic flux experienced at the two locations. Forexample, as described above in relation to Equations 1 through 16, arotation that approximately aligns the acceleration vectors and themagnetic flux vectors estimated for the two locations may be determinedthat specifies an estimated orientation of the thumb relative to thewrist. The estimates of the accelerations and magnetic flux experiencedat each location may be determined by filtering measurements from therespective accelerometers and magnetometers at the locations. Forexample, the measurements for each sensor may be similarly windowed andaveraged (e.g., by applying a Hamming window lagged with respect to alarge deceleration that triggered the tap event) in a periodcorresponding to the thumb being at rest relative to the rest of thehand on the tap target.

In some implementations, an orientation of the thumb relative to thewrist is determined 1130 based in part on angular rate measurements froma gyroscope attached to the thumb and angular rate measurements from agyroscope attached to the wrist. The angular rate measurements from thegyroscope attached to the thumb may be integrated over a period of timeending during the detected tap event to determine an estimate of anorientation of the thumb during the tap event with respect to areference position (e.g., a rest position of the thumb). The angularrate measurements from the gyroscope attached to the wrist may beintegrated over the same period of time ending during the detected tapevent to determine an estimate of an orientation of the wrist during thetap event with respect to a reference position corresponding to thereference position for the thumb. In some implementations, the referencepositions for the thumb and wrist may be synchronously resetperiodically (e.g., every minute) or upon prompting from a user. Theestimate of orientation of the thumb may be compared to the estimate ofthe orientation of the wrist to determine 1130 an orientation of thethumb relative to the wrist at a time associated with the tap event. Forexample, a rotation may be determined that relates to the two respectiveestimates of orientation.

In some implementations, an orientation of the thumb relative to thewrist may be determined 1130 by combining information about the linearacceleration and angular rate experienced at the two locations. Forexample, acceleration and angular rate measurements for the thumb may beintegrated over a period of time that ends during the tap event todetermine an estimate of the position and/or orientation of the thumbduring the tap event. Similarly, acceleration and angular ratemeasurements for the wrist may be integrated over the same period oftime to determine an estimate of a position and/or orientation of thewrist during the tap event. The position and/or orientation estimatesfor the thumb and wrist may be compared to determine 1130 a orientationof the thumb relative to the wrist. For example, a rotation may bedetermined that relates to the two respective estimates of orientationand a displacement vector may be determined that relates the twoestimates respective estimates of position.

For example, an orientation of the thumb may be determined 1130 by a tapclassification module of device driver running on a computing device. Insome implementations, an orientation of the thumb may be determined 1130by a tap classification module running on a processing device that isattached to the wrist of the user.

A tap target that was touched during a tap event is identified 1140. Aset of characteristics of the tap event may be analyzed to identify 1140which tap target from among a set of configured tap targets has beentapped by the thumb of the user. For example, the tap targets may beconfigured to be located on other fingers of the user (e.g., asdescribed in relation to FIG. 4). The characteristics of a tap event maybe represented as a vector in a feature space and the tap targets may beconfigured by partitioning the feature space into regions associatedwith one or none of the tap targets.

In some implementations, the tap characteristics include an orientationof the thumb (e.g., represented as a quaternion, a Euler angles triple,or an angle weighted axis of rotation). For example, the feature spacefor orientations may be a three-dimensional or four-dimensional space.In some implementations, the tap characteristics include a displacementvector describing the position of the thumb relative to the wrist. Insome implementations, the tap characteristics include an estimate of adeceleration vector associated with the impact of the thumb on the taptarget. In some implementations, different characteristics of the tapmay be combined to form a larger vector in a higher dimensional featurespace. For example, feature vector may include elements of a quaternionrepresentation of a thumb orientation and a three element representationof a displacement vector describing the position of the thumb relativeto the wrist. In this case, the feature space may have seven dimensions.

The feature space may have been previously partitioned based on trainingdata associated with each configured tap target location. For example,the partition may be determined using a nearest neighbor rule applied toa set of cluster centroids for each tap target. In some implementations,the feature space is partitioned based on training data for a largegroup of users. In some implementations, the feature space ispartitioned based on training data for a particular user. The partitionof the feature space may implemented as slicer that maps orientationdata to an identification of one of the configured tap targets or anerror/ignore result.

For example, a tap target may be identified 1140 by a tap classificationmodule of device driver running on a computing device. In someimplementations, a tap target may be identified 1140 by a tapclassification module running on a processing device that is attached tothe wrist of the user.

An orientation of the wrist is determined 1150. In some implementations,an orientation of the wrist relative to the Earth's gravitational fieldis used to distinguish between multiple symbols associated with a taptarget. An estimate of an orientation of an acceleration experienced bythe accelerometer attached to the wrist with respect to the axes of thataccelerometer during the tap event may be determined based onacceleration measurements from that accelerometer. The accelerationexperienced at the wrist during a tap event may be dominated byacceleration caused by the gravitational force of the Earth. Forexample, measurements from the accelerometer attached to the wrist maybe windowed and averaged in a time period corresponding to the tap eventto determine an estimate of the acceleration due to gravity as a vectorrepresented in the basis of the axes of the accelerometer. Estimates ofangles between this gravity vector and the axes of the accelerometer maybe determined as needed to classify the orientation of the wrist withrespect to the gravity vector. For example, one axis of theaccelerometer may be assumed to be approximately parallel to the forearmof the user when the user wears the interface, while the other two axesare perpendicular to the first axis.

For example, a wrist orientation may be determined 1150 by a tapclassification module of device driver running on a computing device. Insome implementations, a wrist orientation may be determined 1150 by atap classification module running on a processing device that isattached to the wrist of the user.

A symbol is assigned 1160 to the tap event. A configured mapping (e.g.,the mapping illustrated in FIGS. 10A-10C) of tap targets to one or moresymbols may be retrieved and used to assign 1160 a symbol to thedetected tap event. In some implementations, multiple symbols areassociated with a tap target and a symbol is selected from among themultiple symbols associated with the tap target based on the orientationof an acceleration experienced by the accelerometer attached to thewrist of the user. This acceleration may be dominated by an accelerationdue to gravity and may provide an estimate of the orientation of thewrist with respect to gravitational field of the Earth. In someimplementations, an estimate of the angle between this acceleration andan axis parallel to the forearm of the user may be used to select asymbol (e.g., as described above in relation to FIGS. 5, 6 and 10A-10C).In some implementations, estimates of one or more angles between thisacceleration and one or more axes that are perpendicular to length ofthe forearm may be used to select a symbol (e.g., as described above inrelation to FIGS. 7, 8 and 10A-10C). Thus, a user may be enabled toindicate a choice from among the plurality of the symbols associatedwith the tap target, by adjusting the angle of wrist and/or the angle ofa forearm of the user with respect to the gravitational force of theEarth during the tap.

Examples of symbols that may be assigned to a tap include analpha-numeric character, a Chinese character, a command for a computingdevice that will cause the computing device to execute an action (e.g.,send a text message or e-mail, answer a call, initiate a call dialingsequence, change slides in a presentation, turn on a radio or an airconditioner, etc.), and meta-keys (e.g., ‘shift’) that change theinterpretation of a concurrent or subsequent tap, among others.

For example, a symbol may be assigned 1160 by a tap classificationmodule of device driver running on a computing device. In someimplementations, a symbol may be determined 1150 by a tap classificationmodule running on a processing device that is attached to the wrist ofthe user.

The symbol may be transmitted, stored, and/or displayed 1170. In someimplementations, the symbol may be transmitted 1170 to another device.For example, a processing device attached to the wrist of the user thatperforms process 1100 may transmit (e.g., via a wireless communicationslink) the symbol assigned to a detected tap to a computing device. Insome implementations, the symbol may be stored 1170. For example,computing device may buffer a sequence of symbols for later access by anapplication or some other thread running on the computing device. Insome cases, the symbols may be stored in non-volatile memory (e.g.,written to a file on a hard-drive when a text file is edited using theinterface. In some implementations, the symbol may be displayed 1170through a display device controlled by the device performing process1100. For example, a symbol (e.g., an alpha-numeric character) assignedto the tap may be displayed by a projector, or an LCD display on amobile device (e.g., a smart-phone or tablet), among other types ofdisplays.

In some implementations, process 1100 may be repeated in a loop toprocess sequences of tap events while the interface is in an activemode.

In some implementations, a user is enabled to turn a palm side of thewrist down to face towards the Earth to enter a cursor manipulationmode, in which acceleration measurements from an accelerometer of theinterface (e.g., an accelerometer attached to the wrist of a user) isused to move a cursor in a virtual space. In this cursor manipulationmode, some thumb tap events may be used to interact with object in thevirtual space. For example, certain tap targets may be mapped to mouseclicks while the user has their palm facing down toward the Earth.

Other manners initiating or terminating a cursor manipulation mode arepossible. For example, a user may execute special gestures (e.g., aparticular thumb tap or another type of gesture). In someimplementations, an interface is used to track three-dimensional spatiallocation of a user's hand and sensor data may be used to determinetwo-dimensional or three-dimensional cursor location. In someimplementations, thumb orientation and/or position in relation to thereference frame on the wrist may be used to determine whether the cursoris engaged or not, so that the cursor can continue to be moved in adirection beyond the reach of the user. For example, the cursor may bedisengaged when the thumb is oriented approximately perpendicular to thelength of the forearm of the user (e.g., held in a thumb-up gesture) andthe cursor may be engaged when the thumb is closer to parallel with thelength of the forearm. Taps and other spatial gestures can then be usedto interact with objects in a virtual space (e.g., replacing mouseclicks or joystick commands). In some implementations, angular ratemeasurements from a gyroscope in an interface may be interpreted toenable a user to rotate objects in virtual space that have been selectedwith a cursor while operating in a cursor manipulation mode. Forexample, a box in three-dimensional virtual space may be selected withthe interface by using a gesture (e.g., a sustained thumb tap) to ‘grab’the object. The box may then be turned or rotated to a desiredorientation based on angular rate measurements from a gyroscope attachedto a hand (e.g., to a thumb or wrist) of the user as the user rotatestheir hand. Another gesture (e.g., removing the thumb from a sustainedtap target location) may be used to ‘release’ the object, leaving it inthe new orientation within the virtual space. In some implementations, auser may be enable to reorient objects in a virtual space based onmeasurements from an accelerometer and a magnetometer in an interface.For example, the orientation of the hand may be determined at differenttimes as a user turns the hand to manipulate an object by analyzingaccelerometer and magnetometer measurements to estimate the orientationof the hand in relation to the background gravitational and magneticfields (e.g., the Earth's gravity and the Earth's magnetic field).Differences in the estimated orientations with respect to the backgroundfields at two more times may be used to update the orientation of anobject in the virtual space in a corresponding manner.

In some implementations, a computing interface described above is pairedwith a display to facilitate user data entry in various ergonomicpositions. For example, the interface (e.g., including one or moresensors worn on the thumb and one or more sensors worn on the wrist) maybe connected to a processing device, such as a computer, which is inturn connected to a display device, such as a projector. A user's bodyis oriented in a comfortable position and the display is positioned inthe user's natural line of sight. For example, a user may lie in a bedor in a reclining chair and view a display projected onto the ceiling.From this position, the user may enter data via the interface andreceive visual feedback via the display. Data entry mode may be startedand stopped by distinctive motions of the hand that are detected withone or more accelerometers in the interface. For example movement of thehand quickly in a circle may be used to indicate the start or stop of adata entry session. The orientation of a plane in which this circularmotion occurs may be used to set a reference earth radius angle for thesession. Earth radius angles estimated during the session may be rotatedby an amount determined by the orientation of the plane of the circularmotion. For example, if the plane of the circular motion is parallel tothe detected gravitational acceleration, then the measured wristorientations may be left unadjusted, while, if the plane of the circularmotion is orthogonal to the detected gravitational acceleration (e.g.,because the user is lying on their back while making the circular motionin front of them), the measured wrist orientations may be rotated by 90degrees to recover orientations in with respect to the user's body. Inthis manner a user's training with one set of wrist orientations may beused while the body is in different positions.

The computing interface may be used to control a wide variety ofcomputing devices in different contexts. For example, a computinginterface including a ring may be used to control one or more processingdevices integrated in an automobile. Gestures (e.g., thumb taps) may beused to control various environmental systems in the automobile. Forexample, a tap target may be mapped to a command for turning an airconditioner on. Another tap target may be mapped to a command forturning a radio on. Another tap target may be mapped to a command forseeking or selecting a radio station. Another tap target may be mappedto a command for unlocking a door, and so on.

In some implementations, measurement data from sensors in an interfaceor other information (e.g., symbols) based on the sensor measurementsmay be transmitted to multiple target processing devices. For example,an interface may be used to broadcast symbols derived from sensormeasurements reflecting user hand motions for display on multipledisplays.

In some implementations, a interface described herein is paired with avisual gesture recognition system. The position tracking capability ofthe interface may be used in conjunction with data from the visualsensors (e.g., camera(s)) to enhance detection of gestures. For example,when the line of sight between the visual sensor and the thumb or theentire hand is obscured, data from the interface may be used tointerpolate gestures.

In some implementations, an interface includes sensor modules orhousings that are detachable from a corresponding fastening article(e.g., a thumb ring or a wrist band). Thus, fastening articles may beinterchangeable. A user may own multiple fastening articles and switchbetween them for various reasons, such as aesthetics or comfort. Forexample, alternative fastening articles may be different colors or somefastening articles may include jewels or other aspects of traditionaljewelery.

Any processes described herein, are not limited to the hardware andsoftware described above. All or part of the processes can beimplemented as special purpose logic circuitry, such as an FPGA (FieldProgrammable Gate Array) and/or an ASIC (Application Specific IntegratedCircuit). All or part of the processes can be implemented, at least inpart, via a computer program product tangibly embodied in non-transientcomputer-readable media, for execution by or to control the operation ofone or more data processing apparatus, such as a computer, specialpurpose microprocessor, or programmable logic components. A computerprogram can be written in any programming language, including compiledor interpreted languages. A computer program can be implemented as astand-alone program or as portion, such as a module or subroutine, of alarger program. A computer program can be deployed to be executed on asingle data processing device or on multiple data processing devices.

Components of different implementations described above may be combinedto form other implementations not specifically described above. Otherimplementations not specifically described above are also within thescope of the following claims.

What is claimed is:
 1. A method comprising: receiving a first set ofacceleration measurements from a first accelerometer that is attached toa thumb of a user; receiving a second set of acceleration measurementsfrom a second accelerometer that is attached to a wrist of the user;receiving a first set of magnetic flux measurements from a firstmagnetometer that is attached to the thumb of the user; receiving asecond set of magnetic flux measurements from a second magnetometer thatis attached to the wrist of the user; detecting, based at least in parton the first set of acceleration measurements, when the thumb is tappedon a surface; when a tap of the thumb is detected, determining, based atleast in part on the first set of acceleration measurements and thesecond set of acceleration measurements, an orientation of the firstaccelerometer relative to the second accelerometer, wherein theorientation is determined based at least in part on the first set ofmagnetic flux measurements and the second set of magnetic fluxmeasurements, in which determining the orientation comprises: filteringthe first set acceleration measurements to determine a first estimate ofacceleration experienced by the first accelerometer while the thumb wasat rest with respect to a hand of the user on the tap target, andfiltering the second set of acceleration measurements to determine asecond estimate of acceleration experienced by the second accelerometerwhile the thumb was at rest with respect to the hand of the user on thetap target; identifying, based on the orientation, a tap target, fromamong a plurality of tap targets located on fingers of the user, thatwas touched during the tap; assigning a symbol associated with the taptarget to the tap; and transmitting, storing, or displaying the symbol.2. The method of claim 1, in which determining the orientationcomprises: filtering the first set magnetic flux measurements todetermine a first estimate of magnetic flux experienced by the firstmagnetometer while the thumb was at rest with respect to the hand of theuser on the tap target; and filtering the second set of magnetic fluxmeasurements to determine a second estimate of magnetic flux experiencedby the second accelerometer while the thumb was at rest with respect tothe hand of the user on the tap target.
 3. The method of claim 1,comprising: receiving a first set of angular rate measurements from afirst gyroscope that is attached to the thumb of the user; and receivinga second set of angular rate measurements from a second gyroscope thatis attached to the wrist of the user; and wherein the orientation isdetermined based at least in part on first set of angular ratemeasurements and the second set of angular rate measurements.
 4. Themethod of claim 1, comprising: determining, based on the second set ofacceleration measurements, an estimate of an orientation of anacceleration experienced by the second accelerometer with respect to theaxes of the second accelerometer during the tap; and wherein assigningthe symbol to the tap comprises selecting the symbol from among aplurality of symbols associated with the tap target based on theorientation of the acceleration experienced by the second accelerometer.5. The method of claim 4, in which the acceleration experienced by thesecond accelerometer is dominated by acceleration caused by thegravitational force of the Earth and the selection is made based atleast in part on an estimate of an angle between the gravitationalacceleration and an axis running approximately parallel to a forearm ofthe user.
 6. The method of claim 4, in which the user is enabled toindicate a choice from among the plurality of the symbols associatedwith the tap target, by adjusting the angle of wrist or the angle of aforearm of the user with respect to the gravitational force of the Earthduring the tap.
 7. The method of claim 4, in which the user is enabledto turn a palm side of the wrist down to face towards the Earth to entera cursor manipulation mode, in which acceleration measurements from thesecond accelerometer are used to move a cursor in a virtual space.
 8. Amethod comprising: receiving a first set of acceleration measurementsfrom a first accelerometer that is attached to a thumb of a user:receiving a second set of acceleration measurements from a secondaccelerometer that is attached to a wrist of the user: receiving a firstset of magnetic flux measurements from a first magnetometer that isattached to the thumb of the user: receiving a second set of magneticflux measurements from a second magnetometer that is attached to thewrist of the user: detecting, based at least in part on the first set ofacceleration measurements, when the thumb is tapped on a surface: when atap of the thumb is detected, determining, based at least in part on thefirst set of acceleration measurements and the second set ofacceleration measurements, an orientation of the first accelerometerrelative to the second accelerometer, wherein the orientation isdetermined based at least in part on the first set of magnetic fluxmeasurements and the second set of magnetic flux measurements, in whichdetermining the orientation comprises: filtering the first set ofacceleration measurements to determine an estimate of a decelerationvector caused by the impact of the thumb with the tap targets;identifying, based on the orientation, a tap target, from among aplurality of tap targets located on fingers of the user, that wastouched during the tap: assigning a symbol associated with the taptarget to the tap: and transmitting, storing, or displaying the symbol.9. The method of claim 8, in which determining the orientationcomprises: filtering the first set acceleration measurements todetermine a first estimate of acceleration experienced by the firstaccelerometer while the thumb was at rest with respect to a hand of theuser on the tap target; and filtering the second set of accelerationmeasurements to determine a second estimate of acceleration experiencedby the second accelerometer while the thumb was at rest with respect tothe hand of the user on the tap target.
 10. The method of claim 8, inwhich determining the orientation comprises: filtering the first setmagnetic flux measurements to determine a first estimate of magneticflux experienced by the first magnetometer while the thumb was at restwith respect to the hand of the user on the tap target; and filteringthe second set of magnetic flux measurements to determine a secondestimate of magnetic flux experienced by the second accelerometer whilethe thumb was at rest with respect to the hand of the user on the taptarget.
 11. The method of claim 8, comprising: receiving a first set ofangular rate measurements from a first gyroscope that is attached to thethumb of the user; and receiving a second set of angular ratemeasurements from a second gyroscope that is attached to the wrist ofthe user; and wherein the orientation is determined based at least inpart on first set of angular rate measurements and the second set ofangular rate measurements.
 12. The method of claim 8, comprising:determining, based on the second set of acceleration measurements, anestimate of an orientation of an acceleration experienced by the secondaccelerometer with respect to the axes of the second accelerometerduring the tap; and wherein assigning the symbol to the tap comprisesselecting the symbol from among a plurality of symbols associated withthe tap target based on the orientation of the acceleration experiencedby the second accelerometer.
 13. The method of claim 12, in which theacceleration experienced by the second accelerometer is dominated byacceleration caused by the gravitational force of the Earth and theselection is made based at least in part on an estimate of an anglebetween the gravitational acceleration and an axis running approximatelyparallel to a forearm of the user.
 14. The method of claim 12, in whichthe user is enabled to indicate a choice from among the plurality of thesymbols associated with the tap target, by adjusting the angle of wristor the angle of a forearm of the user with respect to the gravitationalforce of the Earth during the tap.
 15. The method of claim 12, in whichthe user is enabled to turn a palm side of the wrist down to facetowards the Earth to enter a cursor manipulation mode, in whichacceleration measurements from the second accelerometer are used to movea cursor in a virtual space.
 16. A system comprising: a data processingapparatus; a data storage device storing instructions executable by thedata processing apparatus that upon execution by the data processingapparatus cause the data processing apparatus to perform operationscomprising: receiving a first set of acceleration measurements from afirst accelerometer that is attached to a thumb of a user; receiving asecond set of acceleration measurements from a second accelerometer thatis attached to a wrist of the user; receiving a first set of magneticflux measurements from a first magnetometer that is attached to thethumb of the user; receiving a second set of magnetic flux measurementsfrom a second magnetometer that is attached to the wrist of the user;detecting, based at least in part on the first set of accelerationmeasurements, when the thumb is tapped on a surface; when a tap of thethumb is detected, determining, based at least in part on the first setof acceleration measurements and the second set of accelerationmeasurements, an orientation of the first accelerometer relative to thesecond accelerometer, wherein the orientation is determined based atleast in part on first set of magnetic flux measurements and the secondset of magnetic flux measurements, in which determining the orientationcomprises: filtering the first set acceleration measurements todetermine a first estimate of acceleration experienced by the firstaccelerometer while the thumb was at rest with respect to a hand of theuser on the tap target, and filtering the second set of accelerationmeasurements to determine a second estimate of acceleration experiencedby the second accelerometer while the thumb was at rest with respect tothe hand of the user on the tap target; identifying, based on theorientation, a tap target, from among a plurality of tap targets locatedon fingers of the user, that was touched during the tap; assigning asymbol associated with the tap target to the tap; and transmitting,storing, or displaying the symbol.
 17. The system of claim 16, in whichdetermining the orientation comprises: filtering the first set magneticflux measurements to determine a first estimate of magnetic fluxexperienced by the first magnetometer while the thumb was at rest withrespect to the hand of the user on the tap target; and filtering thesecond set of magnetic flux measurements to determine a second estimateof magnetic flux experienced by the second accelerometer while the thumbwas at rest with respect to the hand of the user on the tap target. 18.The system of claim 16, in which the operations comprise: receiving afirst set of angular rate measurements from a first gyroscope that isattached to the thumb of the user; and receiving a second set of angularrate measurements from a second gyroscope that is attached to the wristof the user; and wherein the orientation is determined based at least inpart on first set of angular rate measurements and the second set ofangular rate measurements.
 19. The system of claim 16, in which theoperations comprise: determining, based on the second set ofacceleration measurements, an estimate of an orientation of anacceleration experienced by the second accelerometer with respect to theaxes of the second accelerometer during the tap; and wherein assigningthe symbol to the tap comprises selecting the symbol from among aplurality of symbols associated with the tap target based on theorientation of the acceleration experienced by the second accelerometer.20. The system of claim 19, in which the acceleration experienced by thesecond accelerometer is dominated by acceleration caused by thegravitational force of the Earth and the selection is made based atleast in part on an estimate of an angle between the gravitationalacceleration and an axis running approximately parallel to a forearm ofthe user.
 21. The system of claim 19, in which the user is enabled toindicate a choice from among the plurality of the symbols associatedwith the tap target, by adjusting the angle of wrist or the angle of aforearm of the user with respect to the gravitational force of the Earthduring the tap.
 22. The system of claim 19, in which the user is enabledto turn a palm side of the wrist down to face towards the Earth to entera cursor manipulation mode, in which acceleration measurements from thesecond accelerometer are used to move a cursor in a virtual space. 23.The system of claim 16, in which determining the orientation comprises:filtering the first set of acceleration measurements to determine anestimate of a deceleration vector caused by the impact of the thumb withthe tap target.
 24. A system comprising: a data processing apparatus; adata storage device storing instructions executable by the dataprocessing apparatus that upon execution by the data processingapparatus cause the data processing apparatus to perform operationscomprising: receiving a first set of acceleration measurements from afirst accelerometer that is attached to a thumb of a user; receiving asecond set of acceleration measurements from a second accelerometer thatis attached to a wrist of the user; receiving a first set of magneticflux measurements from a first magnetometer that is attached to thethumb of the user; receiving a second set of magnetic flux measurementsfrom a second magnetometer that is attached to the wrist of the user;detecting, based at least in part on the first set of accelerationmeasurements, when the thumb is tapped on a surface; when a tap of thethumb is detected, determining, based at least in part on the first setof acceleration measurements and the second set of accelerationmeasurements, an orientation of the first accelerometer relative to thesecond accelerometer, wherein the orientation is determined based atleast in part on first set of magnetic flux measurements and the secondset of magnetic flux measurements; identifying, based on theorientation, a tap target, from among a plurality of tap targets locatedon fingers of the user, that was touched during the tap; determining,based on the second set of acceleration measurements, an estimate of anorientation of an acceleration experienced by the second accelerometerwith respect to the axes of the second accelerometer during the tap;assigning a symbol associated with the tap target to the tap, whereinassigning the symbol to the tap comprises selecting the symbol fromamong a plurality of symbols associated with the tap target based on theorientation of the acceleration experienced by the second accelerometer:and transmitting, storing, or displaying the symbol.
 25. The system ofclaim 24, in which the acceleration experienced by the secondaccelerometer is dominated by acceleration caused by the gravitationalforce of the Earth and the selection is made based at least in part onan estimate of an angle between the gravitational acceleration and anaxis running approximately parallel to a forearm of the user.
 26. Thesystem of claim 24, in which the user is enabled to indicate a choicefrom among the plurality of the symbols associated with the tap target,by adjusting the angle of wrist or the angle of a forearm of the userwith respect to the gravitational force of the Earth during the tap. 27.The system of claim 24, in which the user is enabled to turn a palm sideof the wrist down to face towards the Earth to enter a cursormanipulation mode, in which acceleration measurements from the secondaccelerometer are used to move a cursor in a virtual space.
 28. Thesystem of claim 24, in which determining the orientation comprises:filtering the first set of acceleration measurements to determine anestimate of a deceleration vector caused by the impact of the thumb withthe tap target.
 29. The system of claim 24 in which determining theorientation comprises: filtering the first set magnetic fluxmeasurements to determine a first estimate of magnetic flux experiencedby the first magnetometer while the thumb was at rest with respect tothe hand of the user on the tap target; and filtering the second set ofmagnetic flux measurements to determine a second estimate of magneticflux experienced by the second accelerometer while the thumb was at restwith respect to the hand of the user on the tap target.
 30. The systemof claim 24, in which the operations comprise: receiving a first set ofangular rate measurements from a first gyroscope that is attached to thethumb of the user; and receiving a second set of angular ratemeasurements from a second gyroscope that is attached to the wrist ofthe user; and wherein the orientation is determined based at least inpart on first set of angular rate measurements and the second set ofangular rate measurements.